ggcat
ggcat copied to clipboard
GGCAT keeps panicking about too many open files
I'm trying to run GGCAT on the following dataset: https://ftp.ebi.ac.uk/pub/databases/ENA2018-bacteria-661k/
I had 5T of free disk space and the following flags: -k 31 -m 32 -j 8 -s 1
However, my runs keep panicking on too many open files despite raising ulimit -n
. My first two runs with ulimit -n
of 1024 and 4096 crashed in the following phase:
Started phase: kmers merge prev stats:
Thread panicked at location: libs-crates/parallel-processor-rs/src/memory_fs/file/internal.rs:248:26
Error message: called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }
Backtrace: 0: <unknown>
1: <unknown>
2: <unknown>
3: <unknown>
4: <unknown>
5: <unknown>
6: <unknown>
7: <unknown>
8: <unknown>
9: <unknown>
10: <unknown>
11: <unknown>
12: <unknown>
13: <unknown>
14: <unknown>
15: <unknown>
16: <unknown>
17: <unknown>
18: <unknown>
19: <unknown>
20: <unknown>
21: <unknown>
22: <unknown>
Then I raised the ulimit -n
to 10240, and it crashed in the following phase:
Finished phase: kmers merge. phase duration: 20675.80s gtime: 30013.67s
Started phase: hashes sorting prev stats:
Finished phase: hashes sorting. phase duration: 646.23s gtime: 30659.90s
Started phase: links compaction prev stats:
Thread panicked at location: libs-crates/parallel-processor-rs/src/memory_fs/file/internal.rs:248:26
Error message: called `Result::unwrap()` on an `Err` value: Os { code: 24, kind: Uncategorized, message: "Too many open files" }
Backtrace: 0: <unknown>
1: <unknown>
2: <unknown>
3: <unknown>
4: <unknown>
5: <unknown>
6: <unknown>
7: <unknown>
8: <unknown>
9: <unknown>
10: <unknown>
11: <unknown>
12: <unknown>
13: <unknown>
14: <unknown>
15: <unknown>
16: <unknown>
17: <unknown>
18: <unknown>
19: <unknown>
20: <unknown>
21: <unknown>
22: <unknown>