huggingface
Update outdated dependencies and feature-gate CLI
Making the library a bit more up to date with common dependencies. Unsure whether special testing is required for the cli.
Added a feature-gate to cli but kept it as default. This allows someone only using the library who doesn't wish to use the binary at all to opt out using default-features = false
Also put dirs behind the feature gate http, as it's only used hogether with cached_path in the from_pretrained-flow
Hi Thanks for this PR.
Do you mind splitting the feature gates from the dependencies update in separate PRs ?
It's much easier for me to add the cli-feature gate as I understand the implications more easily.
For dependency updates, we need to run a full chain of tests (in transformers to make sure nothing is broken there).
This typically is done before any release too, but for dependency updates I would also do it before merging on master as it's easier then to figure out what dependency might be breaking things.
Hi Thanks for this PR.
Do you mind splitting the feature gates from the dependencies update in separate PRs ?
It's much easier for me to add the cli-feature gate as I understand the implications more easily.
For dependency updates, we need to run a full chain of tests (in
transformersto make sure nothing is broken there). This typically is done before any release too, but for dependency updates I would also do it before merging on master as it's easier then to figure out what dependency might be breaking things.
No problem, I did leave the dirs feature gating in since it's only used in a single place in the codebase, which is already behind an existing feature gate. That PR is here https://github.com/huggingface/tokenizers/pull/960