tox-pyenv icon indicating copy to clipboard operation
tox-pyenv copied to clipboard

CircleCI 2.0 compatible?

Open manihamidi opened this issue 7 years ago • 5 comments

Am I correct that the comment on README regarding "the list of python versions that are pre-installed in the CircleCI build environment (as of 09/27/2017)", does not apply in CircleCI 2.0?

Any recommendations on usage for CircleCI 2.0?

Thanks

manihamidi avatar Feb 22 '18 00:02 manihamidi

@manihamidi Good question. I'll look into this soon. I've never ported a circle config from v1 to v2. Are you familiar with this? If so, could you put up a PR which changes the circle.yml of this project to version 2.0?

https://github.com/samstav/tox-pyenv/blob/master/circle.yml

tox-pyenv uses tox-pyenv for testing, so this project itself can be used to test usage of tox-pyenv in a circle 2.0 environment.

stavxyz avatar Feb 22 '18 17:02 stavxyz

thanks @samstav. I just started playing with Circle and so jumped straight into 2.0. I'll attempt a port of tox-pyenv's yml to 2.0 soon... stay tuned for a PR.

manihamidi avatar Feb 22 '18 21:02 manihamidi

TL;DR I think that Circle 2.0 and its use of containers makes pyenv redundant.

My understanding is that there are two main options for running tox on Circle 2.0:

  • With "thin containers" - meaning that one container per Python version would be used and a test matrix would be run on Circle. Each container could still do a number of tasks with tox like "test", "lint", etc, but tox would no longer be responsible for changing Python version.

    As per this post on the discussion forum: "On CircleCI 2.0 we recommend using different container images for different language versions." - I think this tallies with the "thin containers" approach.

    This approach means that at test time, Circle will orchestrate the change of containers. At dev time, some scripting or tool (maybe like dox) would run each container in turn. This means that the single version of Python available will be used by each container meaning no need for pyenv.

  • With a single "fat container" - this more closely emulates what happens on Circle 1. One container with multiple versions of Python installed and tox switches between each one. I think that using the "fat container" approach is easier when upgrading from v1.

    The benefit of the "fat container" is that a container instance can be built to deliver the exact functionality required for a test suite. If there isn't a version of Python available that is required, then add it and rebuild the container.

    I've been experimenting with kpndigital/tox:master as a fat container on Circle 2.0 and no pyenv is required.

As an experiment, I created a branch, but it's currently RED :red_circle: . These were the steps that I took:

  • Reformatted the circle.yml file for version 2. This should probably be moved to .circle/config.yml to match the expected location for Circle 2.

  • Pushed - Build 3 fails because pyenv is not available in the kpndigital/tox container.

  • Removed the self install (pip install -U .) and the pyenv call.

  • Pushed - Build 4 gets to the tox -v --recreate step but fails because of an error in setup.py.

jamescooke avatar Apr 05 '18 14:04 jamescooke

@jamescooke thanks for digging on this.

Just to get this out of the way-- I think even when the build gets a bit further, you'll end up running into the same thing I did over here: https://github.com/samstav/tox-pyenv/pull/15#issuecomment-377388091


That said.... I think maybe if you add universal_newlines=True to the check_output call you could get around the TypeError: sequence item 3: expected str instance, bytes found

stavxyz avatar Apr 06 '18 21:04 stavxyz

@samstav Here's the build with universal_newlines turned on https://circleci.com/gh/jamescooke/tox-pyenv/5 - it gets to the same popen stuff as you've seen before I think.

jamescooke avatar Apr 09 '18 09:04 jamescooke