File exceeds memory mapped reader max buffer size.

[brvpuyve]

Hi,

I am trying to process large .mgf files (approximately 10Gb), but unfortunately SearhGUI crashes. The memory assigned to Java was already increased to 250 Gb, but no luck. Any suggestions on how to resolve this?

Best, SearchGUI 4.1.1 log.txt Bart

[mvaudel]

Hi Bart,

If nothing is suspicious about these mgf files (acquisition and peak-picking are ok), the low hanging fruit is to split your mgf files. I can fix our handling of the spectra to accommodate larger MS files, but I don't expect to have the time to look into it before October.

Apologies for the inconvenience,

Marc

[kbarylyuk]

Hi Marc and the team, I am running in the same error with a roughly 5.5 Gb mzML file, and I am wondering if there has been any development going on to enable handling larger files. I am not sure if a workflow with converting the data to MGF files and then splitting them is going to be optimal in my setting, but I am happy to give it a try if you could kindly provide a bit more detailed guidance on how to do it. Thank you! Konstantin PeptideShaker log:

Tue Jan 23 14:09:31 CET 2024: PeptideShaker version 3.0.5.
Memory given to the Java virtual machine: 34359738368.
Total amount of memory in the Java virtual machine: 218103808.
Free memory: 163524640.
Java version: 21.0.1.
1714 script command tokens
(C) 2009 Jmol Development
Jmol Version: 12.0.43  2011-05-03 14:21
java.vendor: Microsoft
java.version: 21.0.1
os.name: Mac OS X
memory: 68.0/167.8
processors available: 10
useCommandThread: false
java.io.IOException: File exceeds memory mapped reader max buffer size.
	at com.compomics.util.experiment.io.mass_spectrometry.cms.CmsFileWriter.addSpectrum(CmsFileWriter.java:109)
	at com.compomics.util.experiment.io.mass_spectrometry.MsFileHandler.writeCmsFile(MsFileHandler.java:224)
	at com.compomics.util.experiment.io.mass_spectrometry.MsFileHandler.register(MsFileHandler.java:150)
	at eu.isas.peptideshaker.gui.NewDialog$24.run(NewDialog.java:1005)

[mvaudel]

Hi Konstantin,

Time is flying! Apologies, I have not been able to prioritize this and am not sure when I can finally refactor this code. In the meantime splitting the file would be the way to go.

Sorry that I cannot help more at this stage,

Marc

[hbarsnes]

I've just released SearchGUI version 4.3.8 that should take care of the problem with the buffer size. If this is not the case, please let me know and I'll reopen the issue.