Clarifying / Updating the Repology Automation Project

[nemani]

So I have started adding Repology IDs to linter bears (there are a lot of them, around 80-90) So for the first evaluation, I am trying to make sure that all the Gem bears' packages are on repology. I would be adding them to Wikidata which is the fastest way of getting data onto repology. My main concern is with the application of the usage of the data, so after a bears package is available on the repology, we can fetch data using HTTP requests to Repology API. But exactly what information are we going to use and what decisions would we take based on this information is something I need help with.

Quoting @margobra8

I think your API calls should be focused on cross-referencing the repositories which host the dependency packages so that they can be liked and referenced recursively. Maybe check the repository URI, the version number so as to decide, in that case, which package to install. You would also need to take into account what to do when a bear requires a specific version of a dependency, but Repology also lists all versions so there should be no problem. I think we should schedule a meeting with all mentors just to see how each one thinks about this and reach a consensus so that we can just steer in the right direction.

I am opening this issue to have a discussion with all the mentors, @jayvdb @margobra8 @underyx @fexpr

[nemani]

This is basically because while finding the correct Repology ID of existing packages or entering data into new Wikidata Items, there are cases where decisions need to be made. I do not have a clear idea of the end goal and thus it becomes difficult for me to make these decisions.

[waldyrious]

while finding the correct Repology ID of existing packages or entering data into new Wikidata Items, there are cases where decisions need to be made.

Hi Arjun. Can you give an example or two of such decisions?

[nemani]

For example for the ruby package rubocop both ruby:rubocop and rubocop exist as metapackage on Repology.

[nemani]

So the final product of the project is supposed to access the API of Repology using the repology id, and get info if a specific version is available in the distro's package manager. We also need a way to access the language's specific package manager like, PyPi or NPM or RubyGems.org

[waldyrious]

For example for the ruby package rubocop both ruby:rubocop and rubocop exist as metapackage on Repology.

Hmm, IIUC those should be merged. But @AMDmi3 is the right person to provide an authoritative answer.

[nemani]

Okay for more such things, I'll add rules the same way and make a PR.

[AMDmi3]

I don't think I fully understand what this is about, but regarding discrepant names like rubocop and ruby:rubocop there's no universal answer or solution. Some repositories tend to prefix modules for specific languages (python, perl, ruby, haskell) to prevent possible conflicts with other packages, some don't. Some behave differently based on whether the subject is more of an application than a module, some don't (and it becomes really messy when the subject is both the application and a module). So such cases unfortunately are and will probably remain common, with only possibility of manual merge.

My own policy for repology is to merge with prefix for modules, and without prefix for applications. Furtunately I haven't encountered ambigous cases yet. This one looks more like an application, so I'm merging this case into rubocup.