memory allocation of 2147483648 bytes failed (that's 2GB!)

[bjeanes]

I tried running it on a directory posted on reddit yesterday:

$ suckit https://icsarchive.org/icsarchive-org/paperback/cookbooks/

memory allocation of 2147483648 bytes failed
zsh: abort      suckit https://icsarchive.org/icsarchive-org/paperback/cookbooks/

I don't understand how it would ever come to try to allocate 2GB of memory. None of the files is that big (and even if they were, they shouldn't be buffered to memory).

I left it running for sometime so it didn't happen until quite late in the game when I was afk. I didn't run with RUST_BACKTRACE so don't have more details at this time, but I can try fetching again to see if I can replicate

[bjeanes]

Ah actually I do have one other bit of information, because my .zshrc outputs timing for any command over 2 seconds:

25:52.17s total, 130.18s user, 196.37s system, 21% cpu

[bjeanes]

I can see from re-running with --versbose that it ascended to a parent directory so I suppose it may have encountered a larger file or has some data structure that is growing with each link and hit some upper bound.

[CohenArthur]

Thanks a lot for the report. We're investigating it

[bjeanes]

Re-running reproduced it at the ~28 minute mark. No backtrace shown even with RUST_BACKTRACE=1 (haven't tried RUST_BACKTRACE=full), but I haven't checked if cargo install strips debug symbols or something. Hopefully you're able to reproduce.

[CohenArthur]

cargo install does strip debug symbols since it's building in release mode. You can compile the project with cargo build if you want them, or simply run cargo run -- <your suckit arguments>. Thanks again, I'll let you know how we're progressing.