latin-bert
latin-bert copied to clipboard
dbamman
tensorflow==2.3.0 ?
Hi, I am trying to install Latin-BERT from scratch and it seems that your requirements (requirements.txt) are too stringent and/or obsolete.
ERROR: Could not find a version that satisfies the requirement tensorflow==2.3.0 (from versions: 2.5.0rc0, 2.5.0rc1, 2.5.0rc2, 2.5.0rc3, 2.5.0, 2.5.1, 2.6.0rc0, 2.6.0rc1, 2.6.0rc2, 2.6.0, 2.6.2, 2.7.0rc0, 2.7.0rc1, 2.7.0)
ERROR: No matching distribution found for tensorflow==2.3.0
Well, too bad ! I upgraded to 2.5.0, but obviously things turned badly again, with numpy.
INFO: pip is looking at multiple versions of tensor2tensor to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of numpy to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of <Python from Requires-Python> to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of future to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of cltk to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of beautifulsoup4 to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of pygame to determine which version is compatible with other requirements. This could take a while.
ERROR: Cannot install -r requirements.txt (line 6), -r requirements.txt (line 7) and numpy==1.18.5 because these package versions have conflicting dependencies.
The conflict is caused by:
The user requested numpy==1.18.5
tensor2tensor 1.15.7 depends on numpy
tensorflow 2.5.0 depends on numpy~=1.19.2
To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict
ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/user_guide/#fixing-conflicting-dependencies
As I am a new-comer in this field, I'd like to have your advice on the way to solve these conflicting dependencies.
A time interval of 16 months is larger than the life-span of a python package ?
Yours,
Philippe.
@PhVerkerk I have updated the installation instructions, dependencies (thanks @todd-cook and PR #5), and also made a note that the current installation to note that LatinBERT is tested on Python 3.7 & 3.8 (and does not work on Python 3.9+). Can you try to install again with this setup and let me know if it works? PJB
Thanks !
I'll try in the next few days, and let you know.
Ph.
Le 26/01/2022 à 21:26, Patrick J. Burns a écrit :
I have updated the installation instructions, dependencies (thanks @todd-cook https://github.com/todd-cook and PR #5 https://github.com/dbamman/latin-bert/pull/5), and also made a note that the current installation to note that LatinBERT is tested on Python 3.7 & 3.8 (and does not work on Python 3.9+). Can you try to install again with this setup and let me know if it works? PJB
— Reply to this email directly, view it on GitHub https://github.com/dbamman/latin-bert/issues/6#issuecomment-1022573771, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACNY23GPGTK5VGAEAEO5JS3UYBKGZANCNFSM5L3EI5YA. Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you authored the thread.Message ID: @.***>
As a matter of fact, the line "tensorflow==2.3.0" is now fine (I moved back to Python 3.8.10 which is the last 3.8.* version with binary installers). But the explosion now occurs in "tokenizers==0.8.1rc2". Something also with "atari-py==0.2.6", but I don't know where it comes from (i.e. not in your requirements.txt)
(Latin_BERT) Philippes-MacBook-Pro:latin-bert-master Philippe$ pip install tokenizers==0.8.1rc2
Collecting tokenizers==0.8.1rc2
Using cached tokenizers-0.8.1rc2.tar.gz (97 kB)
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: tokenizers
Building wheel for tokenizers (pyproject.toml) ... error
ERROR: Command errored out with exit status 1:
command: /Users/Philippe/opt/anaconda3/envs/Latin_BERT/bin/python /Users/Philippe/opt/anaconda3/envs/Latin_BERT/lib/python3.8/site-packages/pip/_vendor/pep517/in_process/_in_process.py build_wheel /var/folders/y_/skp1d_jx6m1cq6twqfllxc080000gn/T/tmpcc69nq1m
[...]
error: cargo failed with code: 101
----------------------------------------
ERROR: Failed building wheel for tokenizers
Failed to build tokenizers
ERROR: Could not build wheels for tokenizers, which is required to install pyproject.toml-based projects
The "[...]" in the code above stands for pages and pages of red prints, probably intended for non-human brains.
Ph.
I don't know if it helps, but in the insane lines shortened with "[...]", there are some more explicit comments (that I am unable to decrypt):
error[E0658]: `if` is not allowed in a `const fn`
--> /Users/Philippe/.cargo/registry/src/github.com-1ecc6299db9ec823/clap-2.34.0/src/app/settings.rs:7:1
|
7 | / bitflags! {
8 | | struct Flags: u64 {
9 | | const SC_NEGATE_REQS = 1;
10 | | const SC_REQUIRED = 1 << 1;
... |
51 | | }
52 | | }
| |_^
|
= note: see issue #49146 <https://github.com/rust-lang/rust/issues/49146> for more information
= help: add `#![feature(const_if_match)]` to the crate attributes to enable
= note: this error originates in a macro (in Nightly builds, run with -Z macro-backtrace for more info)
error[E0658]: `if` is not allowed in a `const fn`
--> /Users/Philippe/.cargo/registry/src/github.com-1ecc6299db9ec823/clap-2.34.0/src/args/settings.rs:6:1
|
6 | / bitflags! {
7 | | struct Flags: u32 {
8 | | const REQUIRED = 1;
9 | | const MULTIPLE = 1 << 1;
... |
28 | | }
29 | | }
| |_^
|
= note: see issue #49146 <https://github.com/rust-lang/rust/issues/49146> for more information
= help: add `#![feature(const_if_match)]` to the crate attributes to enable
= note: this error originates in a macro (in Nightly builds, run with -Z macro-backtrace for more info)
and later
error: aborting due to 2 previous errors
For more information about this error, try `rustc --explain E0658`.
error: could not compile `clap`.
But I was not able to understand the answer to rustc --explain E0658.
Ph.
Thank you for trying the new setup—can you let me know what version of rustc you are running? (I have installed so far with 1.50.0 and 1.58.1.)
As I had no rust compiler this morning, I have installed the one available from https://rustup.rs/
rustc 1.58.1 (db9d1b20b 2022-01-20)
Ph.
Le 27/01/2022 à 17:35, Patrick J. Burns a écrit :
Thank you for trying the new setup—can you let me know what version of rustc you are running? (I have installed so far with 1.50.0 and 1.58.1.)
— Reply to this email directly, view it on GitHub https://github.com/dbamman/latin-bert/issues/6#issuecomment-1023416819, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACNY23GS6N5TIRQPWNXKUZ3UYFX4PANCNFSM5L3EI5YA. Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you were mentioned.Message ID: @.***>
Hey - I know this is old but I ran into this problem and managed to get things running!
Step 1 I needed to do conda create --name latinbert python=3.8 based on @diyclassics ' comment above (lol ur name which I just noticed).
Step 2 I did pip install torch
Step 3 failed for me. Installing all the pip packages at once with pip install -r requirements.txt seemed to send me into some sort of dependency circle. It wasn't clear to me but it failed to install some packages, continued to attempt to resolve dependency issues for a long time (15+ min), and errored on install of many packages. At one point I too was told I didn't have the proper rust install but I think that was bcus python was 3.10. unfortunately can't recall exactly. The solution for me was the pip install package==version for each package in requirements.txt. Despite some warnings, it seems to do fine
Step 4 same as readme
Step 5 I needed to run brew install wget but after that the script worked fine
For clarity's sake, and because I know some people who want to use this will not be very familiar with a lot of these concepts (like conda or packages), these were the commands I ran:
conda create --name latinbert python=3.8
conda activate latinbert
pip install torch
pip install pygame==2.0.0.dev6
pip install beautifulsoup4==4.9.1
pip install cltk==1.0.15
pip install future==0.18.2
pip install numpy==1.18.5
pip install tensor2tensor==1.15.7
pip install tensorflow==2.3.0
pip install tokenizers==0.8.1rc2
pip install transformers==3.1.0
pip install tqdm==4.48.2
pip install Unidecode==1.1.1
python3 -c "from cltk.data.fetch import FetchCorpus; corpus_downloader = FetchCorpus(language='lat');corpus_downloader.import_corpus('lat_models_cltk')"
./scripts/download.sh
python3 scripts/gen_berts.py --bertPath models/latin_bert/ --tokenizerPath models/subword_tokenizer_latin/latin.subword.encoder > berts.output.txt
Hey,
It is old but still pending. As we say in French : "Mieux vaut tard que jamais".
I'll try it soon. Thanks a lot.
Yours,
Philippe.
Le 08/05/2022 à 06:31, Max Marion a écrit :
Hey - I know this is old but I ran into this problem and managed to get things running!
Step 1 I needed to do |conda create --name latinbert python=3.8| based on @diyclassics https://github.com/diyclassics ' comment above (lol ur name which I just noticed). Step 2 I did |pip instal torch| Step 3 failed for me. Installing all the pip packages at once with |pip install -r requirements.txt| seemed to send me into some sort of dependency circle. It wasn't clear to me but it failed to install some packages, continued to attempt to resolve dependency issues for a long time (15+ min), and errored on install of many packages. At one point I too was told I didn't have the proper rust install but I think that was bcus python was 3.10. unfortunately can't recall exactly. The solution for me was the |pip install package==version| for each package in requirements.txt. Despite some warnings, it seems to do fine Step 4 same as readme Step 5 I needed to run |brew install wget| but after that the script worked fine
For clarity's sake, and because I know some people who want to use this will not be very familiar with a lot of these concepts (like conda), these were the commands I ran:
|conda create --name latinbert python=3.8 conda activate latinbert pip install torch pip install pygame==2.0.0.dev6 pip install beautifulsoup4==4.9.1 pip install cltk==1.0.15 pip install future==0.18.2 pip install numpy==1.18.5 pip install tensor2tensor==1.15.7 pip install tensorflow==2.3.0 pip install tokenizers==0.8.1rc2 pip install transformers==3.1.0 pip install tqdm==4.48.2 pip install Unidecode==1.1.1 python3 -c "from cltk.data.fetch import FetchCorpus; corpus_downloader = FetchCorpus(language='lat');corpus_downloader.import_corpus('lat_models_cltk')" ./scripts/download.sh python3 scripts/gen_berts.py --bertPath models/latin_bert/ --tokenizerPath models/subword_tokenizer_latin/latin.subword.encoder > berts.output.txt |
— Reply to this email directly, view it on GitHub https://github.com/dbamman/latin-bert/issues/6#issuecomment-1120348104, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACNY23HCPGE47FKYH47E47DVI47TZANCNFSM5L3EI5YA. You are receiving this because you were mentioned.Message ID: @.***>
Great :) LMK if it works. It worked for my sister on her computer, but another confirmation would be nice. I'd open a pull request to change the README instructions, but another confirmation that this works right would be nice, and having a script that does all the pip installs instead of a requirements.txt just feels wrong. Willing to do that if the authors think its the right choice!
Too bad, it runs into troubles with rustc, with the same message as in my comment dated Jan 27th. I have tried to follow the suggested link ("for more information") https://github.com/rust-lang/rust/issues/49146. But I am unable to understand a word of what is written there. And I didn't see the slightest indication of how to solve the problem. It seems to be a close circle of heavy programmers.
To summarize what I did :
- I started on a "fresh" (but old) Mac (fresh meaning that it has never seen Python, Anaconda or any other species of snakes), installing first Anaconda in the standard way (I am a pedestrian). The reason why I started from scratch is just that this Mac has more space on its HD than my laptop.
- I copied your instructions, one by one, and it was fine till "pip install tensor2tensor==1.15.7" where it failed with atari-py, opencv-python and gevent (in this order). I don't know if it is correlated, but on its way, pip collected first tensorflow-2.7.0 and then the whole series down to tensorflow-2.2.0. I did not count how many attempts it did, but it was quite long.
- As you mentioned a few errors, I went on with tensorflow (OK) but
tokenizers exploded :
The first suggestion was to upgrade pip. I did it but it
changes nothing.
Reading a few lines in the error message, it seems that it
didn't find Rust compiler (unclear why it needs it).
I install rustc (1.60.0) and updated my shell
No hope : "error: could not compile
clap."
Seems unlikely to work. I give up...
Philippe.
Le 08/05/2022 à 22:29, Max Marion a écrit :
Great :) LMK if it works. It worked for my sister on her computer, but another confirmation would be nice. I'd open a pull request to change the README instructions, but another confirmation that this works right would be nice, and having a script that does all the pip installs instead of a requirements.txt just feels wrong. Willing to do that if the authors think its the right choice!
— Reply to this email directly, view it on GitHub https://github.com/dbamman/latin-bert/issues/6#issuecomment-1120482473, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACNY23CCL2EXCQPRS2LQGHLVJAP3PANCNFSM5L3EI5YA. You are receiving this because you were mentioned.Message ID: @.***>
It is much better on Windows 10 ! Except that one has to install Visual C at some point, the "pip chain" is successful. I hadn't enough time to go to the end (last 3 lines), but at least all the packages are there.
Ph.
Le 08/05/2022 à 22:29, Max Marion a écrit :
Great :) LMK if it works. It worked for my sister on her computer, but another confirmation would be nice. I'd open a pull request to change the README instructions, but another confirmation that this works right would be nice, and having a script that does all the pip installs instead of a requirements.txt just feels wrong. Willing to do that if the authors think its the right choice!
— Reply to this email directly, view it on GitHub https://github.com/dbamman/latin-bert/issues/6#issuecomment-1120482473, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACNY23CCL2EXCQPRS2LQGHLVJAP3PANCNFSM5L3EI5YA. You are receiving this because you were mentioned.Message ID: @.***>
Well, I manage to have it working, but quite strange. I am not familiar at all with the language nor the Windows environment...
On my first attempt, the last-but-two line gave the error that the module "cltk" was not there. But it was visible in the "Anaconda Navigator". Strange. No hint of its existence with python3 -c "help('modules')". However, when launching python from the Anaconda Navigator, inputting the command help('modules') shows the cltk-module (and many others that were not there in the terminal, probably all those I had installed this morning with pip). Again, in the "python window", the command itself (every thing between the quotes) worked smoothly. As the command help('modules') told me where the modules are (because of warnings or deprecations), I have added the path to the variable PYTHONPATH, following instructions I have found somewhere (set PYTHONPATH=%PYTHONPATH%;C:/Users/[...]/.conda/...). Then, the cltk-module appears in the list and the last-but-two line works in the terminal. Great !
The last-but-one line gave an error too, but as I had the folder model/latin_bert on a memory stick (I had already downloaded it with my Mac, I do not remember how) I didn't pay much attention to it.
The last line worked immediately, and gives me a file that I have now to understand.
Thanks a lot for having retriggered my interest in the topics.
Le 08/05/2022 à 22:29, Max Marion a écrit :
Great :) LMK if it works. It worked for my sister on her computer, but another confirmation would be nice. I'd open a pull request to change the README instructions, but another confirmation that this works right would be nice, and having a script that does all the pip installs instead of a requirements.txt just feels wrong. Willing to do that if the authors think its the right choice!
— Reply to this email directly, view it on GitHub https://github.com/dbamman/latin-bert/issues/6#issuecomment-1120482473, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACNY23CCL2EXCQPRS2LQGHLVJAP3PANCNFSM5L3EI5YA. You are receiving this because you were mentioned.Message ID: @.***>
Happy to hear that it is working—sorry it was as difficult as it was. I will review this thread and see if the installation instructions can be made clearer. I am going to close this issue—please reopen if other installation issues (esp. with the tensorflow dependency) arise.