Outdated package for urdf_parser_py present on PyPI

[traversaro]

I am surprised we do not have this issue open, as I think we already discussed this about this in the past.

Anyhow, the current situation is that this library is (also) a ROS-independent library and it is packaged as such in conda-forge (), and is packaged with an old (and unmaintained) version in PyPI: https://pypi.org/project/urdf-parser-py/ Just opening the issue to track the fact that downstream users of this parser are looking into forking it as it is not on PyPI, see for example https://github.com/dora-rs/dora/pull/704 or https://github.com/ami-iit/adam/issues/36 .

For reference, this PyPI package seems used, see:

• https://github.com/search?q=urdf-parser-py+path%3A**%2Fsetup.py&type=code
• https://github.com/search?q=urdf-parser-py+path%3A**%2Frequirements.txt&type=code
• https://github.com/search?q=urdf-parser-py+path%3A**%2Fpyproject.toml&type=code

From https://pypistats.org/packages/urdf-parser-py, it seems that the package is downloaded an average of ~1000 times each day.

Possible solutions include:

• Try to claim back the urdf_parser_py name on PyPI
• Package the library under the urdfdom-py name (that is how the package is packaged in ROS) so that it can contain an updated version.

[traversaro]

Something that I never thought is to just contact who uploaded the package to PyPI in the first place, that is https://pypi.org/user/xinsongyan/ .

@xinsongyan sorry to disturb you, just in case do you happen to be the same person that uploaded urdf_parser_py to PyPI. Whould you be interested in updating the version there or sharing the maintenance there so that a updated version is uploaded? Thanks in advance, and please ignore if you are not https://pypi.org/user/xinsongyan/ .

[xinsongyan]

Hi, Silvio,

yes, I uploaded urdf_parser_py some time ago, it might be outdated. I can update it, it this the newest version (https://github.com/ros/urdf_parser_py )?

I can also share the maintenance with you, what is your user name on pypi? [image: image.png]

Thanks, Songyan

On Thu, Nov 7, 2024 at 1:19 PM Silvio Traversaro @.***> wrote:

Something that I never thought is to just contact who uploaded the package to PyPI in the first place, that is https://pypi.org/user/xinsongyan/ .

@xinsongyan https://github.com/xinsongyan sorry to disturb you, just in case do you happen to be the same person that uploaded urdf_parser_py to PyPI. Whould you be interested in updating the version there or sharing the maintenance there so that a updated version is uploaded? Thanks in advance, and please ignore if you are not https://pypi.org/user/xinsongyan/ .

— Reply to this email directly, view it on GitHub https://github.com/ros/urdf_parser_py/issues/85#issuecomment-2462222572, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABUZMOAUW43GRMEO3SKYQ6TZ7NSGTAVCNFSM6AAAAABRLFXA56VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDINRSGIZDENJXGI . You are receiving this because you were mentioned.Message ID: @.***>

[traversaro]

Thanks @xinsongyan, nice to meet you! Thanks a lot, the latest version is https://github.com/ros/urdf_parser_py/releases/tag/1.2.1 .

I can also share the maintenance with you, what is your user name on pypi? [image: image.png]

Thanks a lot, https://pypi.org/user/traversaro/ .

[xinsongyan]

I added you as a maintainer, you can update it on your side. Thanks, Songyan

On Fri, Nov 8, 2024 at 4:34 PM Silvio Traversaro @.***> wrote:

Thanks @xinsongyan https://github.com/xinsongyan, nice to meet you! Thanks a lot, the latest version is https://github.com/ros/urdf_parser_py/releases/tag/1.2.1 .

I can also share the maintenance with you, what is your user name on pypi? [image: image.png]

Thanks a lot, https://pypi.org/user/traversaro/ .

— Reply to this email directly, view it on GitHub https://github.com/ros/urdf_parser_py/issues/85#issuecomment-2465224579, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABUZMOH7DIGJPQF6IA6FQ5DZ7TRX7AVCNFSM6AAAAABRLFXA56VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDINRVGIZDINJXHE . You are receiving this because you were mentioned.Message ID: @.***>

[traversaro]

Thanks a lot!

[traversaro]

To check how to upload to pypi, I first started doing an experiment with TestPyPI, with this commands:

conda create -n testuploadurdfpy python pip twine python-build
conda activate testuploadurdfpy
git clone https://github.com/ros/urdf_parser_py
cd urdf_parser_py
git checkout 1.2.1
python -m build
twine upload --verbose -r testpypi dist/*

this currently fails with error:

Uploading urdfdom_py-1.2.1-py3-none-any.whl
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 20.1/20.1 kB • 00:00 • 21.0 MB/s
INFO     Response from https://test.pypi.org/legacy/:
         400 Bad Request
INFO     <html>
          <head>
           <title>400 'License :: OSI Approved :: BSD' is not a valid classifier. See
         https://packaging.python.org/specifications/core-metadata for more information.</title>
          </head>
          <body>
           <h1>400 'License :: OSI Approved :: BSD' is not a valid classifier. See
         https://packaging.python.org/specifications/core-metadata for more information.</h1>
           The server could not comply with the request since it is either malformed or otherwise incorrect.<br/><br/>
         &#x27;License :: OSI Approved :: BSD&#x27; is not a valid classifier. See
         https://packaging.python.org/specifications/core-metadata for more information.


          </body>
         </html>
ERROR    HTTPError: 400 Bad Request from https://test.pypi.org/legacy/
         Bad Request

After fixing this with https://github.com/ros/urdf_parser_py/pull/88, the package is now available at https://test.pypi.org/project/urdfdom-py/ . So, it seems that to upload the new package, we need nevertheless to upload it under a new name urdfdom-py. Unfortunately there is no easy way to deprecate a PyPI package and point users to a new package (see https://discuss.python.org/t/adding-a-mechanism-to-deprecate-a-published-project/13937/, https://github.com/pypa/packaging.python.org/issues/883), but we could always upload a simple metapackage to urdf_parser_py that installs the new urdfdom-py package.

[traversaro]

Recap

Let's do a bit of recap. @clalancette if you like, feel free to ignore the rest of the conversation and just check this recap, and provide any opinion/feedback!

First of all, it is important to clarify that this repo is named urdf_parser_py, but it contains the urdfdom-py Python package that itself contains the urdf_parser_py Python module. This is a bit complicated, but it is important to point out as it means that if we want to package this library in PyPI, we need to do it under the urdfdom-py Python package name.

Problem

There is an outdated version of the urdf_parser_py python module uploaded as the urdf_parser_py uploaded on PyPI, and many users use it (~1000 downloads per day, see the OP for more description).

Possible Actions Points

• Option 1: Do nothing, i.e. do not package the package on PyPI. A fair point may be "this is a ROS 2 package, and if we allow people to also download it from PyPI, this could lead to confusion of users installing it from both apt ROS 2 packages and PyPI, leading to confusing errors". However, there is already a urdf_parser_py package on PyPI, so I am afraid that ship has already sailed.

• Option 2: A third-party (i.e. different from the urdf_parser_py maintainers, for example me) upload the updated urdfdom-py to PyPI, independently from the repo itself. This in theory does not require any permission or action from urdf_parser_py maintainers (at the moment just @clalancette), but in practice I would like to make sure that the maintainers do not object this. This is a bit similar to what we did for the conda-forge packaging of this repo in https://github.com/conda-forge/staged-recipes/pull/21870 and https://github.com/conda-forge/urdfdom-py-feedstock, but the situation here is a bit more "strange" as typically PyPI uploads are managed by the library maintainers directly, not via third-party packages as it happens in most other non-language specific distros.

• Option 3: We integrate PyPI upload of the urdfdom-py package via GitHub Actions in this repo. This has the advantage that new tags will be automatically uploaded to PyPI, but also requires to be fully onboard on this. If it is helpful, I would be happy to help as a maintainer of the repo in this case.

If we go for Option 2 or Option 3, we can also upload a urdf-parser-py meta-package that only installs urdfdom-py to ensure that anyone that installs urdf-parser-py gets the updated version of the urdf_parser_py python module, but that is quite orthogonal.

[haixuanTao]

Option 2 and 3 sounds good IMO!

What about starting with option 2 and mark the third party as experimental. See how it works and then either keep it as is or migrate to this repo?

[traversaro]

Option 2 and 3 sounds good IMO!

What about starting with option 2 and mark the third party as experimental. See how it works and then either keep it as is or migrate to this repo?

Sure! I would still like to have a green light from @clalancette even for option 2, but after ~2 months I guess we can proceed independently (as anyhow the broken package is there, and is not going away).

What do you mean by "mark the third party as experimental"? Is that something we can do on PyPI?

[haixuanTao]

So reactive :') . Just within the pypi readme. Mark urdfdom-py as a third-party distribution which is experimental and that it may be a breaking dependencies depending on their ROS installation ( I would expect ) and so this third-party pypi package should be avoided in conjunction of ROS2.

And ultimately, if people complains for whatever reason we can yank/delete the project.

How does that sound?

[traversaro]

Good idea! Are you interested to be maintainer in the repo, just to reduce the bus factor?

[haixuanTao]

Sure thing I can try to help, but I'm catching up with the urdf spec sheet and I'm not proficient at all in it

[traversaro]

Sure thing I can try to help, but I'm catching up with the urdf spec sheet and I'm not proficient at all in it

Sure, but the repo will just contain the action to upload the wheel, no actual code at all.

[traversaro]

@mjcarroll this issue may be also related to https://github.com/ros/urdf_parser_py/pull/87#issuecomment-2757356139 .

[traversaro]

I uploaded the urdfdom-py package on PyPI at https://pypi.org/project/urdfdom-py/#history . I will do some basic tests, and if that work fine I will also upload a dummy urdf_parser_py package that will just depend on urdfdom-py, to ensure that urdf_parser_py users will stop using an outdated version of the library, and finally close this issue.

[traversaro]

Great, a second after uploading I found a bug, import urdf_parser_py.xml_reflection fails as the lxml and yaml dependencies are not correctly listed in the setup.py . However, as I do not think the xml_reflection subpackage is particularly used, and anyhow users can add manually the lxml and yaml dependencies, I do not think this is particularly problematic.

[traversaro]

Great, a second after uploading I found a bug, import urdf_parser_py.xml_reflection fails as the lxml and yaml dependencies are not correctly listed in the setup.py . However, as I do not think the xml_reflection subpackage is particularly used, and anyhow users can add manually the lxml and yaml dependencies, I do not think this is particularly problematic.

Related PR: https://github.com/ros/urdf_parser_py/pull/91 .

[traversaro]

I will do some basic tests, and if that work fine I will also upload a dummy urdf_parser_py package that will just depend on urdfdom-py, to ensure that urdf_parser_py users will stop using an outdated version of the library, and finally close this issue.

Some months have passed, I had uploaded urdfdom-py to PyPI, but I did not uploaded a dummy urdf_parser_py that just depends on the latest urdfdom-py, and as expected most downloads still go to the old urdf_parser_py:

• https://pypistats.org/packages/urdfdom-py
• https://pypistats.org/packages/urdf_parser_py