singularity-hpc icon indicating copy to clipboard operation
singularity-hpc copied to clipboard

The Python module `rpds`, a dependency of SHPC, does not get installed.

Open dipietrantonio opened this issue 10 months ago • 3 comments

Describe the bug

When installing SHPC 0.1.30 the dependency rpds does not get installed alongside it, causing errors at runtime.

spack@joey-01:/software/setonix/2025.03/pawsey/software/shpc/registry/quay.io/pawsey/pytorch> shpc uninstall pytorch
Traceback (most recent call last):
  File "/software/setonix/2025.03/pawsey/software/shpc/bin/realshpc", line 8, in <module>
    sys.exit(run_shpc())
             ^^^^^^^^^^
  File "/software/setonix/2025.03/pawsey/software/shpc/lib/python3.11/site-packages/shpc/client/__init__.py", line 556, in run_shpc
    main(args=args, parser=parser, extra=extra, subparser=helper)
  File "/software/setonix/2025.03/pawsey/software/shpc/lib/python3.11/site-packages/shpc/client/uninstall.py", line 10, in main
    from shpc.main import get_client
  File "/software/setonix/2025.03/pawsey/software/shpc/lib/python3.11/site-packages/shpc/main/__init__.py", line 10, in <module>
    from .settings import Settings
  File "/software/setonix/2025.03/pawsey/software/shpc/lib/python3.11/site-packages/shpc/main/settings.py", line 22, in <module>
    import jsonschema
  File "/software/setonix/2025.03/pawsey/software/shpc/lib/python3.11/site-packages/jsonschema/__init__.py", line 13, in <module>
    from jsonschema._format import FormatChecker
  File "/software/setonix/2025.03/pawsey/software/shpc/lib/python3.11/site-packages/jsonschema/_format.py", line 11, in <module>
    from jsonschema.exceptions import FormatError
  File "/software/setonix/2025.03/pawsey/software/shpc/lib/python3.11/site-packages/jsonschema/exceptions.py", line 15, in <module>
    from referencing.exceptions import Unresolvable as _Unresolvable
  File "/software/setonix/2025.03/pawsey/software/shpc/lib/python3.11/site-packages/referencing/__init__.py", line 5, in <module>
    from referencing._core import Anchor, Registry, Resource, Specification
  File "/software/setonix/2025.03/pawsey/software/shpc/lib/python3.11/site-packages/referencing/_core.py", line 9, in <module>
    from rpds import HashTrieMap, HashTrieSet, List
ModuleNotFoundError: No module named 'rpds'

I suspect it was not included in the dependency list used by pip to install the needed packages.

dipietrantonio avatar Feb 20 '25 06:02 dipietrantonio

shpc does not use rpds directly (and I'm not sure what it is). From the trace it looks like it's required for jsonschema, so perhaps there was an update to that library that triggered this? I think the bug report would go there, and we might just pin jsonschema to a previously working version in the meantime (while it isn't fixed).

vsoch avatar Feb 20 '25 06:02 vsoch

Well spotted, looks like it comes from referencing. I will open an issue to that repository. Feel free to close this one, thank you!

dipietrantonio avatar Feb 20 '25 06:02 dipietrantonio

Thanks, @dipietrantonio. We can keep this open to follow up if a pin to jsonschema is needed.

vsoch avatar Feb 20 '25 17:02 vsoch