guide icon indicating copy to clipboard operation
guide copied to clipboard

Python conventions improvements ideas

Open se7entyse7en opened this issue 6 years ago • 5 comments

The current status of python conventions is a bit poor. Here are some of the things that I think could be decided and that would help to improve consistency across python projects:

  • supported versions: from some slack discussions it seems to be >=3.5, they could be tested only on travis or using for example tox
  • deps management, packaging and publishing: pip + requirements.txt, pipenv, poetry (I personally like poetry as it also handles packaging and publishing)
  • packaging and publishing: setup.py + twine, poetry
  • virtualenvs compatibility: should the project be compatible only with virtualenvs or maybe even conda?
  • best practices: a common configuration for flake, usage of other tools such as isort for imports sorting

se7entyse7en avatar Mar 14 '19 13:03 se7entyse7en

/cc @vmarkovtsev

smacker avatar Mar 14 '19 14:03 smacker

This is the current state of all our ML projects:

  • We support 3.5+. Generally, we support any Python version equal to or newer than in Ubuntu LTS -2 years ago. That is 16.04 and 3.5. Hence we will drop 3.5 in 2020.
  • We use pip and requirements.txt and are totally happy with them, apart from a few minor complaints. The rationale is that the ML folks have never heard about poerty, myself included, and it is completely not widespread in the Python (ML/DS) community. Anaconda is, but no.
  • setup.py + twine. It works - that's all we need from packaging atm.
  • We do not use virtual environments. We always run shared stuff in Docker.
  • This is something we are investigating. There is some common configuration in the latest projects, and the older ones do not use flake at all in favor of vanilla pycodestyle.
  • Type hinting guide is under development. I am not going to force it over the whole company though.

vmarkovtsev avatar Mar 14 '19 14:03 vmarkovtsev

@vmarkovtsev thank you for the explanation!

se7entyse7en avatar Mar 14 '19 14:03 se7entyse7en

We support 3.5+. Generally, we support any Python version equal to or newer than in Ubuntu LTS -2 years ago. That is 16.04 and 3.5. Hence we will drop 3.5 in 2020.

This seems sensible. We should add this already, since it's our current minimum.

deps management, packaging and publishing: pip + requirements.txt, pipenv, poetry (I personally like poetry as it also handles packaging and publishing)

We use pip and requirements.txt and are totally happy with them, apart from a few minor complaints.

Using pipenv for applications would make sense to me. This has been discussed a few times on Slack, but never got a clear picture.

Type hinting guide is under development. I am not going to force it over the whole company though.

It can be a good recommendation for everyone though.

smola avatar Mar 19 '19 15:03 smola

Type hinting guide is under development. I am not going to force it over the whole company though.

I think that really helps to understand the code faster, and it should be a really very nice to have at least for "public" methods. Just as an example we started using type hinting in lookout-sdk and along with sphinx docstring makes the code easy to follow (example).

se7entyse7en avatar Mar 20 '19 15:03 se7entyse7en