Index proxying/mirroring mode

[spinus]

One of the most time consuming things in pypi is checking index and packages versions. Do you plan to introduce something like mirroring index (with refreshing) to avoid connecting to pypi with installing each package?

[tzulberti]

HI.

There is a configuration flag: SHOULD_USE_EXISTING (https://flask-pypi-proxy.readthedocs.org/en/latest/installing.html#configuration) that won't check for versions on pypi. The problem when setting that value to True, is that you won't be able to get new versions of the package.

I don't know if that was what you were looking for.

[spinus]

Not exactly, but near.

For example we have local proxy like:

package==1 
package==2  (this version is downloaded)
package==3

but now author published package==4

I think to mix mirror for index and proxy (as you did) for downloading. So:

pip install package==1 # works - downloads from pypi and cache it (proxy)
pip install package==2 # works - local
pip install package==3 # works - downloads from pypi and cache it(proxy)
pip install package==4 # does not work (local index is not fresh)

Now, for example once a day flash-pypi-proxy refresh the index and we make package==4 familiar.

Current connection flow is (if I understand correctly):

pip install package -> connection to proxy -> connection to pypi index
                            -> connection to proxy (check download cache) -> connection to pypi package (download)

What do you think about somethink like that:

pip install package -> connection to proxy (check local mirrored index) -X-> avoid additional pypi request
                            -> connection to proxy (check download cache) -> connection to pypi package (download)
+ updating index from time to time

[tzulberti]

That is a great solution. I will start working on that.

[spinus]

Nice. Thank you.