Distribution on Pip

[juancarlospaco]

Hello, very interesting project, I have some libs for Python coded on Nim and uploaded to PyPI.

Maybe you can do the same with pipelines, feel free to ask any questions, should be really easy.

I link the repos so you can use it as an example implementation. My Python PyPI packages dont need Nim to install and run.

pip install faster_than_requests

https://github.com/juancarlospaco/faster-than-requests#faster-than-requests

• Faster simple alternative to urllib.requests.

pip install faster_than_csv

https://github.com/juancarlospaco/faster-than-csv#faster-than-csv

• Faster simple alternative to csv, some people doing data science seems to love this one.

pip install faster_than_walk

https://github.com/juancarlospaco/faster-than-walk#faster-than-walk

• Faster simple alternative to os.walk().

All code is there, so feel free to check how its done if you want to do the same.

Uploaded to PyPI with the setup.py on that repo, with python setup.py sdist --formats=zip upload.

The repo also provides a Dockerfile for people wanting to try it without installing it.

Copy the manifest.in, setup.py and setup.cfg from that repo, edit to add your personal/project data, edit the file list on the setup.py, and that should be enough.

If you are not interested on distributing via pip feel free to close this issue. :cat:

[calebwin]

Sounds interesting - I get how faster_than_walk and faster_than_csv expose specific functions to use in Python; I'm not sure I see how or what Pipelines would expose to a Python programmer.

The intended way of using Pipelines is you build components in Python and then write a .pipeline script that can be executed in parallel. This provides a neat "separation of concerns" for someone designing a data pipeline. What kind of functionality would be useful to expose? or is there some other reason to distribute on Pip that I'm missing?

[juancarlospaco]

I was not asking for Python API, I meant if you want to also distribute it via pip for Python devs, if you are not interested, feel free to close the issue. I use nimble anyways haha, but Python people tends to use pip. :slightly_smiling_face: