ml-benchmarks
ml-benchmarks copied to clipboard
scikit-learn
One step
According to the Joel test, we should be able to make the build in one step. Ideally, we would want the documentation to be updated automatically after running the benchmark scripts (and not have to copy/paste values into a .py file and run that). This PR is a first step in that direction.
These changes support easy generation of timing graphs. It works by using a common interface to print the timing result and scores, which also stores these values and finally saves them to file (using pickle). Once the benchmark file (for a task, e.g. "svm") has been run on all the datasets, the user (and in the future the buildbot) can run $ python plot.py task to generate the plot in one step.
To use, run $ python bench_svm.py arcene $ python bench_svm.py madelon $ python plot.py svm
The next step is to integrate the scores and timings directly into the documentation.
The timings and the scores rst table is generated automatically, which I think is much more convenient for updating the documentation.