SolverBenchmark.jl icon indicating copy to clipboard operation
SolverBenchmark.jl copied to clipboard

export profile data

Open d-monnet opened this issue 2 years ago • 16 comments

d-monnet avatar Oct 12 '23 14:10 d-monnet

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Files Coverage Δ
src/profiles.jl 100.00% <100.00%> (ø)

:loudspeaker: Thoughts on this report? Let us know!.

codecov[bot] avatar Oct 12 '23 14:10 codecov[bot]

Hi @d-monnet ! Thank you for the PR.

Connected to #140 (comment) do you think we would also need such function in BenchmarkProfiles.jl ?

I'm not sure a .csv export function would be needed in BenchmarkProfiles.jl which is to me too low level for the generic user, but it would require almost zero work since it would be pretty much a copy of what's in get_profile_solvers_data and export_profile_solvers_data. Let me know if you want me to do that.

Could you also think about some unit tests for this new functionality?

Sure, I'll add the tests. Is it possible to create a file and load it through githut actions though ?

d-monnet avatar Oct 12 '23 22:10 d-monnet

Ok, so let's do it in BenchmarkProfiles.jl first then, if that's ok for you.

I see it is possible for the file. I could imagine the following process: call the function, test that the file exists, and remove the file. Let me know if that doesn't work.

tmigot avatar Oct 13 '23 07:10 tmigot

Here is the PR for csv export in BenchmarkProfiles.jl: https://github.com/JuliaSmoothOptimizers/BenchmarkProfiles.jl/pull/95

d-monnet avatar Oct 13 '23 13:10 d-monnet

Hi @d-monnet , let me know when I can review this. Do you need a new release of BenchmarkProfiles.jl for this?

tmigot avatar Oct 18 '23 19:10 tmigot

Hi @d-monnet , let me know when I can review this. Do you need a new release of BenchmarkProfiles.jl for this?

Sure, still working on it but it should be done (tests included ;) ) by the end of the week. A new release of BenchmarkProfiles.jl would avoid code duplication in this package, if it's ok. However I think there is a compatibility issue for Tables versions between BenchmarkProfile.jl and SolverBenchmark.jl...

d-monnet avatar Oct 18 '23 20:10 d-monnet

There is a BenchmarkProfile.jl 0.4.4 with the new feature. What is the compatibility issue?

tmigot avatar Oct 19 '23 11:10 tmigot

There is a BenchmarkProfile.jl 0.4.4 with the new feature. What is the compatibility issue?

The new release of BenchmarkProfiles.jl requires Tables.jl 1.1 and PrettyTables.jl in SolverBenchmark.jl requires an older version.

d-monnet avatar Oct 19 '23 14:10 d-monnet

Hi @tmigot, the PR is ready for review.

d-monnet avatar Oct 19 '23 15:10 d-monnet

I think the failed doc is unrelated to this. The CI Julia 1.6 failure, however is. It is probably because a file is not correctly removed ? The shortcut would be to modify https://github.com/JuliaSmoothOptimizers/SolverBenchmark.jl/blob/main/test/runtests.jl and tests pkgbmark.jl before the rest.

tmigot avatar Oct 23 '23 13:10 tmigot

Ok, so we will have to investigate further to see what is the problem with Julia 1.6... Otherwise it is good for me. Thank you!

tmigot avatar Oct 23 '23 19:10 tmigot

Based on that branch I have developed a function to export the profile figures as tikz, which I prefer to include in my papers rather than the .png exported by performance_profile(). Let me know if/how you want me to add this contribution. Not sure how to do it properly until this PR is merged...

d-monnet avatar Oct 24 '23 16:10 d-monnet

Based on that branch I have developed a function to export the profile figures as tikz, which I prefer to include in my papers rather than the .png exported by performance_profile(). Let me know if/how you want me to add this contribution. Not sure how to do it properly until this PR is merged...

Would that involve additional packages?

tmigot avatar Oct 24 '23 20:10 tmigot

Based on that branch I have developed a function to export the profile figures as tikz, which I prefer to include in my papers rather than the .png exported by performance_profile(). Let me know if/how you want me to add this contribution. Not sure how to do it properly until this PR is merged...

Would that involve additional packages?

No, the whole figure source code is "hand made" from profiles plots points.

d-monnet avatar Oct 24 '23 20:10 d-monnet

Ok, so why not. Maybe we do the same and add it to BenchmarkProfiles first ?

tmigot avatar Oct 24 '23 21:10 tmigot

Ok, so why not. Maybe we do the same and add it to BenchmarkProfiles first ?

Here is the PR: https://github.com/JuliaSmoothOptimizers/BenchmarkProfiles.jl/pull/104

d-monnet avatar Oct 25 '23 19:10 d-monnet