moabb
moabb copied to clipboard
Save results immediately
Closes #419
Instead of waiting for the evaluation of a whole dataset to finish before saving the results, we could save them as soon as they are computed. Beneficial in case the evaluation fails in the middle.
Pending questions:
- [ ] Can that create issues in case of parallel access to the hdf5 file?
- [ ] What is the computational overhead due to accessing the hdf5 more often?
@bruAristimunha the code change should be quite simple I think (see my commits) but I don't have time now to test the parallel case and benchmark in the standard case.
Okay, but now we lost all the results in the tutorials with this modification. We will need to debug this issue more to get it all green.
I wonder if you could focus a little more on this problem in the CI, and then we could create a more specific test to work on the parallelization and overhead access issue. @PierreGtch
I see it fails while testing the benchmark
function. By default, benchmark uses -1 jobs. I think it might already be an issue with the parallel access to the file...
When I tested in local, the tests/evaluations.py were working fine