Feature request: Lock file support for applications

[sbor23]

I would like to formally request the feature to support lock files for reproducible application builds.

I'm aware this has come up a couple of times, but it might be handy to track it in a fresh issue. What has changed? The documentation [1] states that this feature is blocked by pips lack of resolving capabilities. However it seems that pip install --report --dry-run should do what is needed [2][3][4].

Just noticed that this also open as a discussion, so leaving this here so it's linked: https://github.com/pypa/hatch/discussions/355

This should resolve a list of previous requests:

• https://github.com/pypa/hatch/issues/47
• https://github.com/pypa/hatch/issues/217
• https://github.com/pypa/hatch/issues/360

Reference:

• https://hatch.pypa.io/latest/meta/faq/#libraries-vs-applications
• https://github.com/pypa/pip/pull/10771
• https://github.com/pypa/pip/pull/10748
• https://pip.pypa.io/en/stable/reference/installation-report/

[ofek]

This is actually helpful, thanks!

[hynek]

JFTR as a datapoint: As someone who recently got an ARM computer and needs to deploy to amd64, I’ve switched from pip-tools to PDM for the sole reason that it can do cross-platform lockfiles. With my current setup (and I know I’m not alone), I wouldn’t be able to use a locking system that doesn’t provide for this (pls don’t say Docker).

[funkyfuture]

here are just some random thoughts about this topic:

• it should be a non-default option.
• shall there be an extra subcommand for these installations?
• or would the lockfile-contents get "projected" into a package's metadata?
• imo this feature only makes sense when an installation would strictly stick to the lockfile and verify the dependency packages' hashes, not like …
• could the feature be provided by a plugin?

[Kludex]

This is actually helpful, thanks!

Does this mean that is a wanted feature for hatch?

[ofek]

I have said forever that locking will be supported eventually and that hasn't changed

[Kludex]

I have said forever that locking will be supported eventually and that hasn't changed

Given your tone, should I have already been aware of it? 🤔

[ofek]

I didn't intend for that to have a negative tone!

I wouldn't expect anyone to be aware of anything since I haven't taken the time to write a roadmap yet but there are various issues and discussions stating that this is planned. Originally I was waiting for an official PEP, but that didn't happen, then I was waiting for pip to gain the dry run/resolve flag but that turned out to not be sufficient for cross-platform use cases, so now the plan is to add a locking method to environment plugin types with the built-in first one being pip-tools

[orsinium]

If you use optional dependencies and features, you should already be able to use pip-tools to generate the lock file and feed it into hatch using PIP_CONSTRAINT env var. The only problem I see so far is that automatic dependencies installation won't be triggered when only the lock file changes.

I'm going to give a try this integration. Since I use Taskfile, I should be able to force trigger dependency installation on lock file changes.

[juftin]

I've been working on writing a hatch plugin to use pip-compile as a Lockfile generator. pip-compile is a tough library to integrate with though because it's meant to be used exclusively as a CLI and not a python library so there's not a clean API to use (see this issue).

The plugin is still in early days but I plan to continue working on it here: https://github.com/juftin/hatch-pip-compile. Hopefully it's useful for someone looking to integrate pip-tools directly with hatch.

[jordantshaw]

Has there been any progress made on this. This is really the only thing keeping me from switching over to hatch.

[ofek]

The plugin mentioned in the preceding comment by Justin is the best that is possible right now and Hatch could not provide anything better in the immediate future as the only improvement would be a single cross-platform file format and that is a significant amount of effort.