scancode.io icon indicating copy to clipboard operation
scancode.io copied to clipboard

`populate_purldb` pipeline fails

Open JonoYang opened this issue 7 months ago • 0 comments

I have a project that I created using the inspect_packages pipeline. I ran the populate_purldb pipeline on it and it ended with a failure status. Looking at the error log, I see that this exception is being raised:

'latest' is not a valid <class 'univers.versions.SemverVersion'>

Traceback:
  File "/opt/scancodeio/scanpipe/pipelines/__init__.py", line 214, in execute
    step(self)
  File "/opt/scancodeio/scanpipe/pipelines/populate_purldb.py", line 48, in populate_purldb_with_discovered_dependencies
    purldb.populate_purldb_with_discovered_dependencies(
  File "/opt/scancodeio/scanpipe/pipes/purldb.py", line 326, in populate_purldb_with_discovered_dependencies
    unresolved_packages = get_unique_unresolved_purls(project)
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/scancodeio/scanpipe/pipes/purldb.py", line 279, in get_unique_unresolved_purls
    vers = range_class.from_native(extracted_requirement)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/scancodeio/venv/lib/python3.12/site-packages/univers/version_range.py", line 394, in from_native
    comparator=comparator, version=vrc(version_constraint)
                                   ^^^^^^^^^^^^^^^^^^^^^^^
  File "<attrs generated init univers.versions.Version>", line 7, in __init__
    self.__attrs_post_init__()
  File "/opt/scancodeio/venv/lib/python3.12/site-packages/univers/versions.py", line 87, in __attrs_post_init__
    raise InvalidVersion(f"{self.string!r} is not a valid {self.__class__!r}")

I tried modifying https://github.com/nexB/scancode.io/blob/main/scanpipe/pipes/purldb.py#L280 to catch the InvalidVersion exception. Running the pipeline again, I get this exception:

type object 'NotImplementedError' has no attribute 'constraints'

Traceback:
  File "/opt/scancodeio/scanpipe/pipelines/__init__.py", line 214, in execute
    step(self)
  File "/opt/scancodeio/scanpipe/pipelines/populate_purldb.py", line 48, in populate_purldb_with_discovered_dependencies
    purldb.populate_purldb_with_discovered_dependencies(
  File "/opt/scancodeio/scanpipe/pipes/purldb.py", line 334, in populate_purldb_with_discovered_dependencies
    unresolved_packages = get_unique_unresolved_purls(project)
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/scancodeio/scanpipe/pipes/purldb.py", line 291, in get_unique_unresolved_purls
    if not vers.constraints:
           ^^^^^^^^^^^^^^^^

I put in some print statements and the package that this is happening to is:

pkg_type:  composer namespace:  components name:  jquery extracted_requirement:  >=1.5
vers:  <class 'NotImplementedError'>

The ComposerVersionRange needs to have the from_native method implemented.

@keshav-space or @AyanSinhaMahapatra :

  1. What should we do when the requirement is just latest?

  2. What should a composer package requirement look like? I am assuming that the methods from npm that handles this stuff would work here.

JonoYang avatar Jun 25 '24 23:06 JonoYang