silero-models icon indicating copy to clipboard operation
silero-models copied to clipboard

Packaging and PyPI releases

Open espdev opened this issue 2 years ago • 9 comments

Hello,

Thank you for your hard work.

Is there any chance of getting installable Python package from PyPI for the project?

For example, it might look like this for installing STT models with PyTorch:

pip install silero-models-stt[torch]

This would be very handy for using the models in the production projects and environments.

espdev avatar Feb 13 '22 07:02 espdev

Hi,

Since the majority of this is actually done by torch.hub (it just pulls the repo behind the scenes and loads the model via utils in the hubconf.py file) and the models mostly depend only on PyTorch, we decided to invest our time elsewhere. Though we of course thought about making some packages.

And if someone is using a model in a dockerfile - he can just use torch.hub's cache and / or just freeze the necessary model.

So, since this is the first time anyone actually requests this, I would rather prefer if the community would lend a hand and do some PR using GitHub Actions.

snakers4 avatar Feb 13 '22 07:02 snakers4

Thanks for the reply.

I also mean, it would be nice to have a some high-level API to use your models as a library. Such API would hide low-level implementation details and the ML framework (pytorch, tensorflow, etc). A user will just install the package and use some convenient API to use STT/TTS functionality. The user may not even know anything about pytorch and torch.hub for example.

espdev avatar Feb 13 '22 13:02 espdev

@snakers4 I can pick up on this and make a PR. Fine by you ?

Also I would need some additional details like organization name, minimal python version, initial version scheming, max line length of your coding style, etc.

Abhi011999 avatar Feb 15 '22 14:02 Abhi011999

Hi @Abhi011999 Many thanks for your offer

As for your questions (and some more):

Also I would need some additional details like organization name,

I believe there will be some more fields to fill, probably let's discuss them in DM, if we decide to proceed? Do you use telegram?

minimal python version

I believe since now PyTorch 1.10 official images are provided with python 3.7 Starting with 3.7 makes sense

initial version scheming

I believe nothing fancy, like v0.1-alpha, v0.1-beta, 0.2, 0.3, etc

max line length of your coding style, etc.

We typically do not adhere to any strict rulesets (i.e. we mostly just use linters without pre-commit formatting hooks or similar) We generally follow standard linter conventions and try making code 140 chars wide

I do not believe it would make sense to include a lot of tests, but probably 1 simple test per each model, like in torch.hub would suffice

As for other things, PyTorch by default uses conda, but since there are no binary dependencies (basically we just need torch, torchaudio for STT and omegaconf for reading configs) maybe it does not matter whether to use conda or pip.

After all, the core of the library would be just an init.py file with proper namespace, since more or less everything is already packaged.

omegaconf can be removed in favour of yaml (which is in standard library I believe), but that would require replacing all of the dot notations with .get, i.e. conf.param with conf.get('param')

snakers4 avatar Feb 15 '22 17:02 snakers4

Hi @snakers4 , Yeah let's discuss this in the TG DM. I have messaged you.

Abhi011999 avatar Feb 16 '22 21:02 Abhi011999

Hi @snakers4

Made a PR #127.

Review and suggest changes.

Thanks.

Abhi011999 avatar Feb 22 '22 20:02 Abhi011999

We are refactoring with @Abhi011999 and something may break If anyone notices some broken examples, please report here

snakers4 avatar Feb 24 '22 16:02 snakers4

Keeping track of the PRs:

https://github.com/snakers4/silero-models/pull/129 https://github.com/snakers4/silero-models/pull/128 https://github.com/snakers4/silero-models/pull/127

snakers4 avatar Feb 28 '22 07:02 snakers4

  • Updated the pip package several times, mostly still testing
  • Fixed a PyTorch version bug https://github.com/snakers4/silero-models/commit/da053c9de3f03fe393248f07e2306c5bbb003c25
  • Each update needs a commit like this https://github.com/snakers4/silero-models/commit/eb948ad55d0441533aceea35cb9e0118aaf42537
  • Action is triggered on tag CREATION, which poses some issues ... if I created a tag erroneously, lol

snakers4 avatar Jun 06 '22 18:06 snakers4