proseg Overflow Problem in CosMx 6000plex

First, thank you very much for developing such a great tool!

Now we encountered overflow problem in CosMx 6000plex. This is error log.

proseg --cosmx Proseg_tx_file.csv.gz -t 10 Using 10 threads Read 644060072 transcripts 760384 cells 6718 genes Estimated full area: 126778810 Full volume: 1267788000 Using grid size 176.02158. Chunks: 7665 00:00:00 ############################################################ | log-likelihood: -5218807000 | assigned: 624538431 / 644060072 (96.97%) | non-background: (89.58%)thread 'main' panicked at /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/arrow2-0.18.0/src/array/physical_binary.rs:208:41: called Result::unwrap() on an Err value: Overflow note: run with RUST_BACKTRACE=1 environment variable to display a backtrace

Jul 03 '24 04:07 SanoKyohei

Thanks for trying proseg. Just to verify, because I fixed a similar bug a while ago, is this with the latest version (1.0.5)?

Also, I haven't yet extensively tested proseg on higher plex. I think it may run into some problems in some cases, but I'm working on improving the model for these cases.

Jul 04 '24 00:07 dcjones

Thank you for your response. I was using version 1.0.3, so I will try with 1.0.5. By the way, are the differences between 1.0.3 and 1.0.5 only related to bug fixes and changes in output format? Is there a possibility of creating a batch effect by analyzing data output from both versions 1.0.3 and 1.0.5 together?

Jul 04 '24 03:07 SanoKyohei

No guarantees, but I think this bug was fixed with 1.0.5. There have been no changes to the actual model between 1.0.3 and 1.0.5, so it should not introduce batch effects.

Jul 04 '24 05:07 dcjones