rust-pip icon indicating copy to clipboard operation
rust-pip copied to clipboard

Implemented Warehouse Pypi API Call

Open Allstreamer opened this issue 3 years ago • 19 comments
trafficstars

  • Added log 0.4 Crate
  • Added serde 1.0 Crate
  • Added serde_json 1.0 Crate
  • Created pypi.rs
  • Created request_package_info function
  • Created PypiData struct
  • Added request_package_info example to main.rs

Implementaition of what was mentioned in

  • #13
  • #17

Allstreamer avatar Aug 08 '22 20:08 Allstreamer

Please change target to the develop branch

John15321 avatar Aug 08 '22 20:08 John15321

Solved the merge conflicts and retargeted to develop

Allstreamer avatar Aug 08 '22 20:08 Allstreamer

Fixed Formating & Linting

Allstreamer avatar Aug 08 '22 20:08 Allstreamer

When writing the name pypi with capital letters make sure its "PyPI" not Pypi or PyPi that also includes structs

John15321 avatar Aug 09 '22 07:08 John15321

Please add docstrings to all structs, methodsz functions etc

John15321 avatar Aug 09 '22 07:08 John15321

Please add tests. Both usage unit tests in the example section of the docstrings and negative cases in a test module and some integration tests to check if the cli works. I know it may be complicated but that's why we don't have a lot of written code rn. I have to research mock modules etc we want to use so try ur best for now

John15321 avatar Aug 09 '22 07:08 John15321

Additionally i don't see any issues connected to this PR that explain what's being done here. So please create one and link it

John15321 avatar Aug 09 '22 07:08 John15321

Should now be mergable

Allstreamer avatar Aug 09 '22 09:08 Allstreamer

Had a name mixed up where i was getting info for numpy but the test was named pytorch

Allstreamer avatar Aug 09 '22 09:08 Allstreamer

You could have your PyPIPackageInfo look like this since we wont need most of the info that the JSON API gives:

struct PyPIPackageInfo {
    version: String, // Or a tuple of MAJOR, MINOR, PATCH assuming that every python package uses semver
    digest: String,
    url: String,
    python_version: String, // Again, semver tuple can be used here
    size: u8,               // Can be helpful in progress bars
    filename: String,
}

Kiwifuit avatar Aug 09 '22 13:08 Kiwifuit

Not every package uses semver (see package "AWRS" which has the version 2019.8.14.1152)

Allstreamer avatar Aug 09 '22 14:08 Allstreamer

I'll probably rewrite it that way but first i need to find out everything that rust-pip could need

Allstreamer avatar Aug 09 '22 14:08 Allstreamer

This is looking better and better. Don't worry we are not in a rush. It's not like anyone's gonna rewrite pip in rust before us. And personally due to multiple private reasons i cannot do us too much on this project so chill

John15321 avatar Aug 09 '22 14:08 John15321

Yeah, don't worry too much! I'm still a long ways away from being happy with the code in this pull request!

Still need a way to calculate python version requirements from strings such as: >=2.7,!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*

A way of managing package versions and release information, Since not every release that's listed in the api has information.

and so on!

So i'll be crafting away at this pull request for a while!

Allstreamer avatar Aug 09 '22 14:08 Allstreamer

#[derive(Debug, Serialize, Deserialize, PartialEq)]
pub struct PyPIPackageRelease {
    version: String,
    filename: String,
    md5_digest: String,
    required_python: String,
    size: u32,
    url: String,
}

#[derive(Debug, Serialize, Deserialize, PartialEq)]
pub struct PyPIPackageData {
    name: String,
    stable_version: String,
    releases: Vec<PyPIPackageRelease>
}

Something like this but with the versions replaced with some kind of struct is what i'm thinking of @Kiwifuit

Allstreamer avatar Aug 09 '22 15:08 Allstreamer

I remember finding crates that solve server for you maybe you can find something. Also please mark this PR as draft if that's the case

John15321 avatar Aug 09 '22 15:08 John15321

I remember finding crates that solve server for you maybe you can find something. Also please mark this PR as draft if that's the case

Sadly PyPi allows arbitrary strings and not just semver

Allstreamer avatar Aug 09 '22 15:08 Allstreamer

I've been looking into how pip manages versions and it looks like they have two systems:

  • Their own version system in this format: int, Tuple[int, ...], PrePostDevType, PrePostDevType, PrePostDevType, LocalType
  • And a legacy system which allows abitrary strings:
class LegacyVersion(_BaseVersion):
    def __init__(self, version: str) -> None:
        self._version = str(version)
        self._key = _legacy_cmpkey(self._version)

        warnings.warn(
            "Creating a LegacyVersion has been deprecated and will be "
            "removed in the next major release",
            DeprecationWarning,
        )

Since as it says above that the legacy system will be removed with the next release i think i should just implement their current version system and throw an Err when reciving an abitrary string

Allstreamer avatar Aug 09 '22 16:08 Allstreamer

Yeah in general i don't plan on supporting anything legacy. This is supposed to be bloatless pip for the future

John15321 avatar Aug 09 '22 16:08 John15321