fix!: normalize Python depName in every manager

[not7cd]

Changes

Adds normalized packageName to every Python manager

Context

Related to #16012

Documentation (please check one with an [x])

• [ ] I have updated the documentation, or
• [ ] No documentation update is required

How I've tested my work (please select one)

I have verified these changes via:

• [ ] Code inspection only, or
• [ ] Newly added/modified unit tests, or
• [ ] No unit tests but ran on a real repository, or
• [ ] Both unit tests + ran on a real repository

[Churro]

Wouldn't that break package rules (and replacement presets) that now match the name with -_.? Not sure why replacing it in the pypi datasource wouldn't be enough...

[not7cd]

Is there a way to make this change non-breaking? Would this involve some sort of normalization of the configuration itself?

[not7cd]

There are several problems and they mainly comes down to comparasion logic.

1. vulnerability alerts are case sensitive: for example Pillow is returned with its non-normalized name, but it may often exist in requirements in its normalized form as pillow. This results in Renovate not creating PRs.
2. Package name matching it may be cumbersome to match for both package_name and package-name or Django and django.

This is evident with pip-compile as lock files always have normalized names.

[rarkins]

https://github.com/renovatebot/renovate/issues/28546 will make matchPackageNames case-insensitive, hopefully in v38

[not7cd]

As per documentation. There are more valid names that can lead to rules not matching.

https://packaging.python.org/en/latest/specifications/name-normalization/

This means that the following names are all equivalent:

• friendly-bard (normalized form)
• Friendly-Bard
• FRIENDLY-BARD
• friendly.bard
• friendly_bard
• friendly--bard
• FrIeNdLy-._.-bArD (a terrible way to write a name, but it is valid)

[rarkins]

Then we should generate appropriate matchPackagePatterns instead of matchPackageNames

[not7cd]

What do you mean by that? We as a manager? Where such logic should exist?

Still the simplest way is to compare normalized names. If I understand correctly packageName is such canonical name while depName is used fo display.

[not7cd]

I would prefer normalizePythonPackageName as this is general spec, as per doc linked earlier.

How this issue relates to #28551? It's desirable to include this PR with another breaking change?

Also, a workaround #28214 could be reverted/refactored with this PR.

[not7cd]

Should https://github.com/renovatebot/renovate/pull/27733/commits/7babd040765c76e4c362991193197cb652f86e51 be moved to separate PR? Maybe normalizePythonDepName should be moved to a different module, as it is now widely used?

[not7cd]

This may break configs where they match for some_package or some.package.

[not7cd]

Can we get a resolution on that PR? I'm getting a lot of conflicts on force-pushes to v38 and less will to resolve them every time.

[jeroenmaas]

We're running into similar (or same) issue. https://github.com/renovatebot/renovate/discussions/29230 Great work on this not7cd. Hope this could be resolved soon.

[egilewski]

It was suggested in https://github.com/renovatebot/renovate/discussions/29381#discussioncomment-9641345 that this PR probably will fix an issue of uppercase Pillow 10.2.0->10.3.0 upgrade not considered a security one. That discussion's bug seems to be the same as described in https://github.com/renovatebot/renovate/pull/28214#issuecomment-2066621576

[not7cd]

My objections to changes related to this change request are still unresolved. I'm pointing this out as they are now hidden in this thread.