csu-hmc
Created a working unit test for leg2d.m.
@tvdbogert I've reworked the test for leg2d.m to be an actual unit test, but it is currently failing because leg2d does not reproduce the reference results that were checked in to the repository.
One source of the error is that we likely did not update the reference results when fixing issue #70.
It almost seems like the reference results were generated by the orginal test.m file. Do you have some independently computed reference results for this data?
Here is a plot of the differences:
Actually, I have some HBM data you shared with me a long while back. I think I can get that working.
Ok this took forever, but it is potentially usable. I've included a set of data from your HBM paper that you gave me early on. It has filenames like CAREN_Trial04.txt. It includes the markers and ground reaction loads plus the HBM output (except for joint forces). So I setup the test to compare to that data as a "gold standard". It works except that the results aren't as close as I'd hoped.
- The 2D inverse dynamics can fail for big marker dropout gaps which this data has for the right GTRO marker. I found a section of the data that doesn't have too large of gaps.
- It took me a a while to realize that this trial has bad left ground reaction loads (I'd discovered that a year ago, but this caught me a again today) and that the force plate numbers are switched wrt to our treadmill.
Here is the current comparison of HBM to the 2D method in the master branch:
Questions:
- How close should we expect the 2D inverse dynamics to be to the 3D?
- Do you have a better set of "gold standard" data that has the input and output for HBM?
- Do you have the joint forces for this data set?
How close should we expect the 2D inverse dynamics to be to the 3D?
These differences between 2D and 3D model are not unexpected. In fact,
there is no "true answer" to the inverse dynamic problem when using
actual human data. It will depend on which model is used/assumed.
Furthermore, if there is noise in the data and a finite sampling rate,
there is only so much you can do in the inverse analysis.
My original test for the leg2d code was just to make sure that the answer stayed the same through revisions of the code. And if the code improved such that results changed, we would have to update the reference result (which we apparently forgot to do).
- Do you have a better set of "gold standard" data that has the input and output for HBM?
Back in 1996 I posted some inverse dynamic benchmark data, with a true answer and Matlab code. Data came from a 2D simulation with known torques. If I do the inverse dynamics with the same 2D model, no noise on the data and high sampling rate, I can exactly reproduce the known input. It's still there: http://isbweb.org/data/invdyn/
Do you have the joint forces for this data set?
HBM does not calculate joint forces. It was done with Kane's equations so the workless constraint forces are not solved. I made an extension to solve the knee forces, but that's still only the knee. HBM was originally designed with full-body coupled dynamics (rather than Winter-style segment-by-segment analysis) so it could potentially work with incomplete force plate data, but in practice this was never used.
http://isbweb.org/data/invdyn/But don't spend too much time, I think what you have now is fine as a unit test. Perhaps passing the test can be defined as: overall RMS error in joint moments (when compared to HBM) is less than some constant. But consider: comparing to HBM is no better than comparing to results of one version of the code that you once decided was "correct".
Ton
On 3/30/2015 10:31 PM, Jason K. Moore wrote:
Ok this took forever, but it is potentially usable. I've included a set of data from your HBM paper that you gave me early on. It has filenames like |CAREN_Trial04.txt|. It includes the markers and ground reaction loads plus the HBM output (except for joint forces). So I setup the test to compare to that data as a "gold standard". It works except that the results aren't as close as I'd hoped.
- The 2D inverse dynamics can fail for big marker dropout gaps which this data has for the right GTRO marker. I found a section of the data that doesn't have too large of gaps.
- It took me a a while to realize that this trial has bad left ground reaction loads (I'd discovered that a year ago, but this caught me a again today) and that the force plate numbers are switched wrt to our treadmill.
Here is the current comparison of HBM to the 2D method in the master branch: compare2hbm https://cloud.githubusercontent.com/assets/276007/6911261/0c8f366c-d713-11e4-8788-1880f0321b7b.png
Questions:
- How close should we expect the 2D inverse dynamics to be to the 3D?
- Do you have a better set of "gold standard" data that has the input and output for HBM?
- Do you have the joint forces for this data set?
— Reply to this email directly or view it on GitHub https://github.com/csu-hmc/GaitAnalysisToolKit/pull/132#issuecomment-87904290.
My original test for the leg2d code was just to make sure that the answer stayed the same through revisions of the code.
Good to know that it was just a regression test. That's fine and we can possibly keep it that way.
Back in 1996 I posted some inverse dynamic benchmark data, with a true answer and Matlab code.
Awesome, I'll try that out and see if I get a cleaner match.
Perhaps passing the test can be defined as: overall RMS error in joint moments (when compared to HBM) is less than some constant. But consider: comparing to HBM is no better than comparing to results of one version of the code that you once decided was "correct".
Sure, I'll get this in shape with that in mind. I see two types of tests that are useful for these kinds of models:
- regression: make sure your future code changes don't change the original results (unless you discover your original results were wrong)
- independent comparisons: i.e. do I get the exact same (or similar enough) answer as other people's methods? In general, we should assume that if many people do calcs independently and some percent match, then the largest # of matchers are likely to have the correct answer. But there could of course be a systematic error that the large # of matchers are all doing wrong.
When I have time I will look into the performance of my filter when there are large gaps in the data. I'm not sure what you meant by "can fail", can you be more specific? I hope it's still better than filling the gaps with a linear interpolation. Linear interpolation would give zero acceleration in the gap and large acceleration spikes at the beginning and end of the gap. You can see this in Fig 3c in rtfilter-1.pdf. The red curve is linear interpolation followed by conventional digital Butterworth filter.