Relax filename requirements for requirements.txt parsing.

[oliverchang]

There is no strict requirement on requirements.txt as the filename for requirements files. We should be able to detect more cases like these.

[oliverchang]

This is hard to solve perfectly. Some ideas:

• When passed as a --lockfile argument, assume all "*.txt" files are requirements.txt files.
• When scanning the filesystem, we can make this a bit stricter and require "requirementstxt" in the filename. Depending on performance, we could potentially also just do all ".txt" files as well.

[oliverchang]

@another-rex should we keep this open, or open another one, to better auto-detect additional requirements files?

[another-rex]

Let's keep this one open

[pulse-mind]

Hello, Thank you for the job done on this tool, it is very useful.

At this time it is working well if the filename is requirements.txt but not if it is called differently. Example:

git:(master) ✗ osv-scanner --config=/Users/pmind/Projects/osv-scanner.toml --lockfile=requirements/requirements.txt
Scanned /Users/pmind/Projects/myproject/requirements/requirements.txt file and found 4 packages
➜  git:(master) ✗ rm -f requirements/requirements.txt
➜  git:(master) ✗ osv-scanner --config=/Users/pmind/Projects/osv-scanner.toml --lockfile=requirements/base.txt
could not determine parser for /Users/pmind/Projects/myproject/requirements/base.txt

Which is common on a django project. A solution was to use requirements parser for all .txt files. Another would be to pass in the args the parser, for example --parser=python_requirements

Also it does not manage -r to include another file Example :

• requirements/production.txt contains :

-r ./base.txt
sentry-sdk==1.15.0  # https://github.com/getsentry/sentry-python
...

And base.txt contains some dependencies use by all the environments (dev, staging, ...).

[G-Rath]

Another would be to pass in the args the parser, for example --parser=python_requirements

You can already do this by putting the parser you wish to use in front of the file path, seperated with a colon:

osv-scanner --config=/Users/pmind/Projects/osv-scanner.toml --lockfile=requirements.txt:requirements/base.txt

See this section of the docs for more

Also it does not manage -r to include another file

Yeah when I initially wrote the requirements.txt parser I hadn't decided how they should be supported given they're another file - i.e. should the packages be reported as being from requirements/production.txt or requirements/base.txt?

Now though I think it should just be treated as if it's all one single file. If you create a new issue for this, I can work on it.

[pulse-mind]

Hi @G-Rath , It works great with the colon :), thanks a lot.

osv-scanner --config=/Users/pmind/Projects/osv-scanner.toml --lockfile=requirements.txt:requirements/base.txt

I created a new issue for nested requirements : https://github.com/google/osv-scanner/issues/252

Best regards, Fred