pyllama icon indicating copy to clipboard operation
pyllama copied to clipboard
verified publisher icon juncongmoo

Downloading get stuck in infinite loop

Open jarimustonen opened this issue 3 years ago • 13 comments

When trying to download the models, I get stuck in some infinite loop.

Screenshot 2023-03-27 at 12 17 59

This repeats once every second until I terminate the program.

Environment:

  • pyllama version: commit 321d475f01c88e179c8a30d68b5281e2caca5b07 (HEAD -> main, tag: v0.0.9, origin/main, origin/HEAD)
  • OS: macOS 13.2.1
  • Hardware: Apple M1 Max
  • After installing the following packages were installed manually:
    • Transformers
      • This was missing from requirements.txt
      • Version 4.27.3
      • Command used pip install transformers
    • Itree
      • Was incompatible with M1 architecture
      • Work-around instructions: https://pypi.org/project/py-itree/
      • Command used: pip uninstall py-itree ; pip install https://github.com/juncongmoo/itree/archive/refs/tags/tag-bf9f3aada064acf3ce4db6fc58ed2e744caee0a3.tar.gz

jarimustonen avatar Mar 27 '23 09:03 jarimustonen

I have this issue too，but the os is WSL2-ubuntu

elven2016 avatar Mar 29 '23 01:03 elven2016

I have the same issue here. @juncongmoo can you help take a look?

Macavity77 avatar Mar 29 '23 20:03 Macavity77

I wish I could reproduce the error above but my run always succeeds. Maybe it is because I don't have an M1 Mac book? Or can anybody create a colab so that I can reproduce the error?

juncongmoo avatar Mar 30 '23 04:03 juncongmoo

Is there a way to add more logging?

@elven2016, what hardware are you using?

@juncongmoo, what is colab and how can I create one?

jarimustonen avatar Mar 30 '23 05:03 jarimustonen

@jarimustonen HP Victus laptop， win11

elven2016 avatar Mar 30 '23 08:03 elven2016

Same problems here. I'm using the M1 chip Macbook Pro, and the downloading was in an infinite loop. I check the size of the file downloaded, and it changes between 0b to 200b, so basically the downloading procedure was just looping.

XiangkunFang avatar Apr 03 '23 20:04 XiangkunFang

Same problem here on a Macbook Pro 14" with M1 Pro 🙋

synchroazel avatar Apr 05 '23 18:04 synchroazel

The problem on Mac is with the underlying download script (associative arrays and nested loop). It's Linux-oriented and should be adopted to MacOS. TBH, I quit trying to fix different issues on Mac last weekend. It was too time-consuming and disappointing. So I stopped and switched back to Linux.

sskorol avatar Apr 08 '23 22:04 sskorol

under MacOS you need to brew install md5sha1sum to get md5sum to pass the verify

mindon avatar Apr 11 '23 19:04 mindon

under MacOS you need to brew install md5sha1sum to get md5sum to pass the verify

I tested this and this won't work.

jarimustonen avatar Apr 13 '23 10:04 jarimustonen

Because it's not the root cause. It never enters this loop.

sskorol avatar Apr 13 '23 12:04 sskorol

under MacOS you need to brew install md5sha1sum to get md5sum to pass the verify

This is worked for M2 Pro too, I also stuck in infinite loop before installation.

Before: Screenshot 2023-05-12 at 4 25 32 AM Screenshot 2023-05-12 at 4 25 56 AM

After: Screenshot 2023-05-12 at 4 22 57 AM

hinofafa avatar May 11 '23 20:05 hinofafa

I am running on colab with gpu and downloading is stuck in infinite loop. Any solutions to solve this problem?

yMayanand avatar May 29 '23 13:05 yMayanand