Richard Harris

Hi @aCoalBall , Can you show some more of the code - specifically how you're handling the `pod5_reader`? Also, are you processing many files, or some very large files?

Hi @aCoalBall , Would you be able to test the following code to ascertain if this is virtual memory issue? ```python pod5_reader = pod5.Reader(pod5_path) pod5_reader._signal_handle._reader = None pod5_reader._signal_handle._reader = pod5_reader._signal_handle._open_without_mmap()...

Hi @davidsilvapires, thanks for the detailed report. Can you report which version of `polars` is installed in each case? Kind regards, Rich

Could you please try pod5 0.3.6 with `polars~=0.19.19`? Kind regards, Rich

Hi @davidsilvapires , I thought we'd fixed this issue in `0.3.6` but apparently not! I'll leave this ticket open until we push a patch. Kind regards, Rich

Hi @diego-rt , We're reworking `subset` which is the underlying process used by `filter` to significantly lower resources and improve performance. Also mentioned here: https://github.com/nanoporetech/pod5-file-format/issues/93#issuecomment-1837267779 We'll hopefully get this out...

Yes you're absolutely correct and these changes will be incorporated to the new design of `filter` and `subset` which will be more stable for large numbers of Inputs / Outputs...

Hi @Jeff-Field , yes you are able to do this with the pod5 tools. Subset is best suited for the use case where you want to create multiple output files...

Hi @hasindu2008, Have you tried following the instructions in the error message? ``` Cargo, the Rust package manager, is not installed or is not on PATH. This package requires Rust...

> @Psy-Fer I already fixed that setup.py install to be pip install . on the 21st of December and sent a PR here [Psy-Fer/blue-crab#10](https://github.com/Psy-Fer/blue-crab/pull/10). So that is not the issue....