Add dependency check to project step runs
Goal
Ensure dependencies specified in requirements.txt are installed. Notes:
- No tests have been added yet. Dedicated tests would be nice, but we'd need to mess with the test venv for that - not sure that's worth it?
- Documentation will be added once the code review is through.
Description
Uses pkg_resources to check whether all dependencies can be imported without issues. Note that this doesn't check the complete dependency tree - theoretically a sub-dependency might be missing or mismatched without this check noticing.
Types of change
New feature.
Checklist
- [x] I confirm that I have the right to submit this contribution under the project's MIT license.
- [x] I ran the tests, and all new and existing tests passed.
- [ ] My changes don't require a change to the documentation, or if they do, I've added all required information.
Have you looked at the timing for this yet?
Have you looked at the timing for this yet?
Not yet, wanted to confirm this approach is acceptable before.
If the basic pkg_resources checks are extremely slow we can immediately stop working on it. (I don't think they will be, though.)
If the basic
pkg_resourceschecks are extremely slow we can immediately stop working on it. (I don't think they will be, though.)
So far they felt pretty instantaneous. Will benchmark with more extensive requirement files tomorrow.
Running the validation with this cobbled together list of requirements
scipy
umap
transformers>=3.4.0,<4.3.0
spacy-transformers>=1.0.1,<1.1.0
stanza>=1.3.0,<1.5.0
flair>=0.6.0,<1.0.0
ufal.udpipe
spacy-lookups-data>=1.0.0,<1.1.0
ml_datasets==0.2.0a0
spacy-experimental==0.2.0
typer
numpy
tqdm
wasabi
pydantic>=1.0.0,<2.0.0
fastapi>=0.61.1,<0.62.0
aiofiles
uvicorn>=0.11.6,<0.12.0
spacy-huggingface_hub
plotext
spacy_ray
spacy-streamlit>=1.0.0a0
streamlit
wandb>=0.12.4,<0.20.0
floret>=0.10.1,<0.11.0
natto-py
spacy-streamlit>=1.0.0a0
spacy-lookups-data>=1.0.0,<1.1.0
google-cloud-storage
streamlit
presidio-analyzer==2.2.1
presidio-anonymizer==2.2.1
skweak==0.2.13
hmmlearn==0.2.6
nlpaug
takes - on my local machine - 0.33 seconds in an environment which has all of these dependencies installed and 0.21 seconds in one that's missing most of them. I think that's reasonable.
I think the conflicting dependencies message needs more info.
Displaying the full message from VersionConflict.report() now. This should show streamlit 1.9.2 has requirement click<8.1,>=7.0, but you have click 8.1.3. in the warning message. Can you verify that?
The merge undid some other edits?
The merge undid some other edits?
Weird. But only https://github.com/explosion/spaCy/pull/11226/commits/f58195569c93249d5a1ccbb5fa000706322498e4, right? Re-added the change.