Show which trials went wrong when "running Scores" or "running outputParser"?

[mengyin]

If an error occurs in the "running Scores" or "running outputParser" step, there is just an error message but it doesn't tell me which trial caused that error. It'll be helpful for debugging if dscr can show which trials went wrong...

[stephens999]

so you are getting actual errors where dscr stops, is that right? for example the output parser is run at: output2 = do.call(outputParser$fn,list(output=output1))

maybe we need somethign like try(..) this, and then have a catch that outputs the infile when the try fails?

[stephens999]

@mengyin

[stephens999]

I think we should also add options to reset parser. Having thought about it, I don't like the idea of selective resets, as they will encourage irreproducible results. Instead we should focus on better error handling and reporting

[mengyin]

Yes I had an actual error where dsc stops. Using try() and having a catch seems to be a great idea! Thank you!

I agree that selective resets might encourage irreproducible results... I just felt like it might be time-consuming to rerun all things when my updated codes just affected a few trials... Not sure how to better deal with this problem in general though.

[ramanshah]

I rant about reproducibility a bit here:

https://github.com/stephens999/dscr/issues/43

I agree with Matthew that putting the burden on the user to nimbly and selectively reset subsets of the whole dsc is probably unsafe. How exactly we make it safe to partially rerun computations is, in my eyes, an open (hard) question with room for discussion.