OMCompiler icon indicating copy to clipboard operation
OMCompiler copied to clipboard

Improve result file reading

Open crupp2 opened this issue 9 years ago • 12 comments

Fix calculation of matrix length. Support for mat files >2GB. Support reading incomplete files from interrupted/intermediate solves.

crupp2 avatar Jul 04 '16 02:07 crupp2

Are the test cases run for all platforms? Reading large files is not always cross platform, so this should be tested on all platforms. Is there a test case for reading mat files?

crupp2 avatar Jul 04 '16 02:07 crupp2

@crupp2 the tests run on Linux x86_64 and are then accepted by Hudson. There are also Windows tests running the test suite, and 64-bit OSX. 32-bit Linux runs once per night as well.

sjoelund avatar Jul 04 '16 04:07 sjoelund

@sjoelund Do you know why the mac nightly does not produce a .dmg file? The job succeeds.

hkiel avatar Jul 04 '16 06:07 hkiel

Seems to be that scp is failing without error-message.

sjoelund avatar Jul 04 '16 06:07 sjoelund

I hadn't tested with MSVC, so this should fix that. Are there any other compilers that may be problematic?

crupp2 avatar Jul 05 '16 19:07 crupp2

Please wait on starting a test for this. I was using it with OMEdit to load in results as they were running and it crashed OMEdit. It works fine once the results are finished, but not when they are being updated by a running simulation. It crashes in PlotWindow.cpp (from OMEdit). Any ideas why?

crupp2 avatar Jul 05 '16 22:07 crupp2

OSX also handles large files differently (but it is a 64-bit only OS so using long probably works fine).

sjoelund avatar Jul 06 '16 04:07 sjoelund

What is the status of this pull request? It seems to be a bit outdated. We have already implemented support for writing partial (and still valid) result file as optional feature. What is the status of large result files (>2GB)?

lochel avatar Feb 05 '18 21:02 lochel

See ticket 4380. Basically, I ran out of time to figure out all the nuances of large file support for all the different OS's. I still don't have time. Sorry. It's up to you guys if you want to fix it or dump it.

crupp2 avatar Feb 05 '18 21:02 crupp2

@crupp2 Thanks for your contribution. 👍 @sjoelund Do you know how to fix it for Linux and Mac?

lochel avatar Feb 05 '18 21:02 lochel

For Linux you can add some extra defines and use open64/lseek64. Or using a different define to simply enable 64-bit (if you used off_t everywhere I think).

sjoelund avatar Apr 09 '18 13:04 sjoelund

For OSX, there is no 32-bit OSX since 2012 so it does not really have this issue (the next OSX version will not be able to run 32-bit code at all, it seems).

sjoelund avatar Apr 09 '18 13:04 sjoelund