fsspec
Switch to Poetry, add dependabot and pre-commit for linting. Remove versioneer
POC for #602
I also refactored the tests to use fixtures, which I think is a lot more flexible and will allow us to split the gigantic test file up a bit.
Edit: I also refactored the moto3 server fixture to keep a persistent server running and use unique bucket names. This speeds up the tests a lot.
I am certainly in favour of precommit, and running automatic jobs locally as they do in CI. This should be made uniform across all projects.
I remain to be convinced by poetry...
@efiop , perhaps you have an opinion on applying this kind of thing here and for other fsspec projects?
I remain to be convinced by poetry...
The current setup.py, requirements.txt, test-requirements.txt, manifest.ini setup is definitely considered to be a legacy way of setting up a project now.
Poetry is really simple way to get a consistent project setup with no fuss. The lockfile also guarantees reproducible builds, and combined with a thing like Dependabot means that we'll have a fairly immediate way of knowing if one of our dependencies breaks us without breaking all existing builds. I like it a lot, and while it's not perfect it's probably as close as we've got so far to a simple, fast and consistent CLI (a-la cargo). It intergates well with IDEs, it's fast and it's sane: poetry run [cmd], poetry add [dependency] (--dev), poetry install is about all you really need.
All I can suggest is that we try it and see what happens, and we can always revert if you feel unhappy with it.
I've split out the test refactor into https://github.com/fsspec/s3fs/pull/611 - this is a nightmare to rebase and I think it has a great impact on test readability + speed.
I've split pre-commit out into https://github.com/fsspec/s3fs/pull/612.
I am certainly in favor of precommit, and running automatic jobs locally as they do in CI. This should be made uniform across all projects.
💯 Related to https://github.com/fsspec/community/issues/6 I would start there and then propagate the template into particular projects.
Sorry, I don't have the capacity right now to start it myself, but I would definitely wait (or start) for the template to be introduced and not proceed with this PR here for now. Might be able to come back to that in the mid term future.
+1, let's aim to do this (and pre-commit.ci rather than a action! ) for the next release cycle in ~1month.