file system packages are stored to and retrieved from static cache

[instil-richard]

If one tries to load a package in a requirements file by specifying a path such as:

/my_packages/my_package

static cache is always used as the source for the package, no matter what has changed on the file system, no matter what the version number in the package setup.py; the package process always retrieves the package from static cache and the package contains old code.

Whilst I can (and currently am) disabling static cache useStaticCache: false this means that all other remote packages have to be downloaded each time also, which is unfortunately both slow and not reliable because internets.

The behavior should be that static cache is bypassed for any package installed from the local path every time; this way the current version, whatever that is (as stored on the local file system) is what is packaged and then the package also reflects the same state that development and tests were performed under. Current behavior is such that the source code for what is in the deployment package is not what was tested (which is bad.)

Another option could be to check the setup.py version in the path against the version in static cache first? This forces a developer to increment the package's version number to effect any packaging changes but I'd prefer that static cache just be bypass-able completely because with this option its a workaround but not an error proof solution.

tl;dr right now current behavior makes the use of local packages unreliable without disabling static cache for all packages.

[f00f]

So one would split the requirements into file system and remote. Then install the remote requirements using the cache. And finally, install the file system requirements bypassing the cache.

Is there a reliable way of detecting file system requirements?

[StuartBertram]

It looks like it should be fairly obvious in the requirements.txt:

boto3==1.34.61 ; python_version >= "3.11"
local_package @ file:///home/.../ ; python_version >= "3.11"
vcs_package @ git+ssh://[email protected]/... ;  python_version >= "3.11"

It looks like <package_name> @ file:///... should reliably detect file system requirements.