Distribution on pypi

[calvinmccarter]

This is awesome! It'd be great if you could make this available on PyPI.

[rfeinman]

@calvinmccarter I'm not sure exactly how to do that but do you want to submit a PR?

[calvinmccarter]

Unfortunately I can't do that with a PR. But the steps are actually relatively simple

[rfeinman]

Thanks @calvinmccarter, I agree that it looks relatively straightforward.

I would feel more comfortable uploading an official PyPI package once there are some basic unit tests written for the project. At the moment there are none. I am putting this on my to-do list but I could use some help - would you be interested?

[calvinmccarter]

I can't promise yet, but I'll try to find the time soon!

[calvinmccarter]

@rfeinman - do you think testing is good enough as of #11 for releasing on pip? (I'd like to distribute my own package, which has torchmin as a dependency...)

[rfeinman]

Hi @calvinmccarter

Thank you for the contribution. I would not say that the current tests are sufficient to fulfill the purpose of unit testing--namely, to ensure the robustness and consistency of a software suite across modules and time.

That being said, I'm inclined to go ahead with the pip package anyways. I'm a bit busy with PhD work atm but will try to get to this ASAP.

P.S. - I've also been working on some unit tests, but I haven't merged them yet.

[calvinmccarter]

@rfeinman - sounds good. Also, if you make your package citable via Zenodo, I'll be happy to cite it in a forthcoming paper.

[richinex]

I would like to ask if there is any update on the release via pypi. If not I would like to know how best to use torchmin as a dependency (especially in the distribution of my own library) and how to cite.

[rfeinman]

@calvinmccarter @richinex

I've followed the instructions that @calvinmccarter shared and I now have a preliminary TestPyPI distribution available. The package can be installed as follows:

pip install --index-url https://test.pypi.org/simple/ --no-deps pytorch-minimize

I am working to finalize the full (non-test) pypi distribution.

[richinex]

Thanks @rfeinman,

if I run the above command it successfully installs

#Collecting pytorch-minimize
 #Downloading https://test-#files.pythonhosted.org/packages/49/d3/ba8743747af15e8298a3c06bdb0cfad58ff09ddd8dc3cd8c9f4d5ff47a37/pytorch_minimi#ze-0.0.1-py3-none-any.whl (29 kB)
#Installing collected packages: pytorch-minimize
#Successfully installed pytorch-minimize-0.0.1

Then upon running import torchmin, I get

# ModuleNotFoundError: No module named 'torchmin.optim'

I guess I am doing something wrong or not

[rfeinman]

@richinex - thanks, this was an issue with setup.py and it's now fixed with https://github.com/rfeinman/pytorch-minimize/commit/a252e30f13df65f927f8d4cee67c0975f76909b2. I will need to re-upload the fixed packages to pypi. Will comment shortly when complete.

[rfeinman]

@calvinmccarter @richinex I am running into some difficulties after fixing the bug that @richinex pointed out and I'm wondering if either of you can help.

I fixed the bug and re-built the distribution package without any problems. However, when I try to upload the new package I get an error message that the file already exists:

~ twine upload --repository testpypi dist/*

Uploading distributions to https://test.pypi.org/legacy/
Enter your username: __token__
Enter your password:
Uploading pytorch_minimize-0.0.1-py3-none-any.whl
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 71.9/71.9 kB • 00:00 • 46.1 MB/s
WARNING  Error during upload. Retry with the --verbose option for more details.
ERROR    HTTPError: 400 Bad Request from https://test.pypi.org/legacy/
         File already exists. See https://test.pypi.org/help/#file-name-reuse for more information.

After some reading on the topic I learned that pypi only allows one upload per package version. So I tried updating the package version from 0.0.1 to 0.0.2 and I was then able to upload without error:

~ twine upload --repository testpypi dist/*

Uploading distributions to https://test.pypi.org/legacy/
Enter your username: __token__
Enter your password:
Uploading pytorch_minimize-0.0.2-py3-none-any.whl
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 71.9/71.9 kB • 00:00 • 51.7 MB/s
Uploading pytorch-minimize-0.0.2.tar.gz
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 58.9/58.9 kB • 00:00 • 41.8 MB/s

View at:
https://test.pypi.org/project/pytorch-minimize/0.0.2/

However, when I try to install the new package with pip, the new version is not available:

~ pip install --index-url https://test.pypi.org/simple/ --no-deps pytorch-minimize==0.0.2

Looking in indexes: https://test.pypi.org/simple/
ERROR: Could not find a version that satisfies the requirement pytorch-minimize==0.0.2 (from versions: 0.0.1)
ERROR: No matching distribution found for pytorch-minimize==0.0.2

If I don't specify version 0.0.2 (pip install pytorch-minimize vs. pip install pytorch-minimize==0.0.2) it installs OK but I get the old, buggy package.

[richinex]

I've never uploaded to pypi so I'm at a loss here. Maybe @calvinmccarter can help.

[calvinmccarter]

Most likely there's a problem with 0.0.2. Perhaps the source is in the uploaded build distribution, or the built distribution is in the uploaded source? Strangely, pypi is sensitive to extra files in either of those. I'll be able to look further into this later, if that isn't the issue.

[rfeinman]

@calvinmccarter I'm not sure I understand your message entirely, but I don't think its an issue with the built distribution (.whl) because everything works fine when I install from the .whl locally:

~ pip install --no-deps ./dist/pytorch_minimize-0.0.2-py3-none-any.whl

Processing ./dist/pytorch_minimize-0.0.2-py3-none-any.whl
Installing collected packages: pytorch-minimize
Successfully installed pytorch-minimize-0.0.2

Update:

Installing the new version (0.0.2) from TestPyPI works fine after pip install --upgrade pip. Perhaps there is some kind of delay in adding new package versions to the pypi index?

[rfeinman]

This is all sorted out now. I've uploaded an official working release of v0.0.2 to the pypi index. No need to specify --index-url https://test.pypi.org/simple/ any longer; I've released to production PyPI (as opposed to TestPyPI).

Install:

~ pip install pytorch-minimize
Collecting pytorch-minimize
  Using cached pytorch_minimize-0.0.2-py3-none-any.whl (59 kB)
Requirement already satisfied: scipy>=1.6 in /Users/rfeinman/opt/anaconda3/envs/torchmin/lib/python3.9/site-packages (from pytorch-minimize) (1.7.3)
Requirement already satisfied: torch>=1.9.0 in /Users/rfeinman/opt/anaconda3/envs/torchmin/lib/python3.9/site-packages (from pytorch-minimize) (1.12.1)
Requirement already satisfied: numpy>=1.18.0 in /Users/rfeinman/opt/anaconda3/envs/torchmin/lib/python3.9/site-packages (from pytorch-minimize) (1.20.3)
Requirement already satisfied: typing_extensions in /Users/rfeinman/opt/anaconda3/envs/torchmin/lib/python3.9/site-packages (from torch>=1.9.0->pytorch-minimize) (4.3.0)
Installing collected packages: pytorch-minimize
Successfully installed pytorch-minimize-0.0.2

Test:

~ python -c "import torchmin; print(torchmin.__version__)"
0.0.2

[richinex]

YaaaY. It works