LibreSelery icon indicating copy to clipboard operation
LibreSelery copied to clipboard

Use the Github API to find the URL of dependencies

Open Ly0n opened this issue 3 years ago • 1 comments

One of the most complex parts of LibreSelery is dependency scanning. The performance of this function is quite bad at the moment:

-- pypi: gitpython [==3.1.7]

FOUND global-data-manager: 'https://github.com/SpencerTuft/global-data-manager' [SpencerTuft] -- pypi: pybraries [==0.2.2] FOUND global-data-manager: 'https://github.com/SpencerTuft/global-data-manager' [SpencerTuft] -- pypi: urlextract [==1.0.0] FOUND global-data-manager: 'https://github.com/SpencerTuft/global-data-manager' [SpencerTuft]

This relates to following issue: https://github.com/protontypes/libreselery/issues/98

Even for larger packages, we get bad results for scanning LibreSelery itself. That is why I would suggest removing bibliothecary, Libaries.io, and the many regular expression and parsing we developed with a simple Github API call:

https://docs.github.com/en/graphql/reference/objects#dependencygraphdependency

Extract Upstream URL from Target Repository --> Get Dependencies of Dependency based on a depth parameter -> Gather all URL

Pros:

  • One token less and new users can play with the dependency scanning immediately just with the GitHub token
  • Less Errors in our dependency scanning results
  • LibreSelery will get a pure Python project that will make it much easier to install on different machines.
  • Less complexity in our code

Cons:

  • Less language support: https://docs.github.com/en/github/visualizing-repository-data-with-graphs/about-the-dependency-graph#supported-package-ecosystems
  • Support for other Git platforms will get even more complex.

@cornerman @kikass13 @yarikoptic @krux02 @fdietze Please give some feedback.

Ly0n avatar Sep 08 '20 07:09 Ly0n

@Ly0n that's not exactly true, dependency SCANNING is fine. The dependency lookup (for authors, contributors inside that dependency) is flawed.

as far as im concerned, i don't care. Whatever works in the end is fine. I mentioned in the issue that the https://libraries.io/ api works perfectly. So I would rather keep using it directly instead of using pybraries (who is the real culprit in #98 )

kikass13 avatar Sep 08 '20 13:09 kikass13