Benchmark-Models-PEtab
Benchmark-Models-PEtab copied to clipboard
Adding suggestion of optimizers for benchmark models.
Data needs to be updated with final results.
Thanks, this will be great to have!
I think for maintainability, a TSV table with the "raw" data is preferable. I can add a script that converts it to the markdown table and embeds it into the README. That way contributors can simply edit the TSV table with easy-to-understand "raw data" that they probably have like optimizer, number of starts, number of successful starts, time per start / whatever else, and the script can then compute the overall efficiency if sufficient data is provided.
We would also need a few more details about the optimizer and their hyperparameters (e.g., what is nlopt_42?). Some TSV with columns optimizer_id and description would suffice, we can then think how to present it...
Also would be great to have the fval, gradient, and parameter vector for each model, if you still have that saved somewhere!
Was your parameter vector for each model ever worse (in terms of fval) than the nominal parameters currently in our collection?