SINDy with mixed-integer optimization

[akaptano]

Wes and Dimitris have expressed some interested in getting their algorithm (found here https://arxiv.org/pdf/2206.00176, code found here https://github.com/wesg52/sindy_mio_paper) working in PySINDy (it already adheres to the PySINDy structures). It looks like it works very well, and is quite different in spirit to the other algorithms in PySINDy. One hiccup is that they use gurobi (not open-source), although apparently a reduced academic license is available for free, which can solve small problems (https://www.gurobi.com/documentation/9.5/quickstart_mac/cs_using_pip_to_install_gr.html).

[wesg52]

Small clarification: The reduced license is free for everyone. The academic license is completely unrestricted and is free for academics (which I am guessing is the main user base of PySINDy).

I think as long as all the tests have library dimension less than the reduced license limit (IIRC, 200 is the limit for our formulation) there shouldn't be any problems, and those that want to use MIOSR on larger problems will have to acquire a license (which is very simple as long as you have access to a university wireless network).

[Jacob-Stevens-Haas]

Very cool! In terms of "get their algorithm... working in pysindy", @wesg52 are you envisioning miosr as a standalone PyPI package added as a dependency of pysindy? Or incorporating it fully into the repository?

• A separate library (like we did with derivative) makes it slightly harder to change the interface between pysindy and miosr, but makes all other changes easier for you to make.
• OTOH, if it's part of pysindy, it gets the whole optimizer test suite without having to duplicate code. It's development remains tighter to pysindy, making it easier to adjust the interface (e.g. if we move from sklearn's linear Pipeline to something in tensorflow) and more stable compatability with pysindy.

It's a bit of a theoretical decision, and one I've never dealt with before. Regardless, @akaptano , we might want any reliance on gurobi to be an optional dependency, which would leave the licensing question to the user. That would mean that full capability with gurobi would be installed with pip install pysindy[miosr], but pip install pysindy would allow people who graduate or work in industry to use sindy without running afoul of the gurobi license.

[Jacob-Stevens-Haas]

Along the lines of option 2, I added a miosr branch to the repo in order to check tests. It looks like it fails a few tests. First step would be to run the tests locally and reproduce/debug the errors (I think most are of the zero-argument constructor, e.g. optimizer=MIOSR(), others deal with whether MIOSR admits a max_iter parameter).

Also may want to take a look at enhancing the docstring for the MIOSR class and what specific bad parameter combos you want to add to test_specific_bad_parameters. Finally, would anything else from the miosr repo be worth including into pysindy? There's a lot of other code there, and since miosr.py doesn't import it, I'm not sure of the relationship.

[wesg52]

To be honest, I hadn't considered the former but that could make sense as well. As the owners of the repository, I will defer that decision to you all.

I will say that I don't expect too many major changes to MIOSR, so the value add of additional changes being easier to make is fairly small (it is honestly a pretty simple model, gurobi is doing most of the heavy lifting). This is especially true because if users want to do something fancy, they can just access the underlying gurobi model directly to implement whatever custom constraints or modeling logic is appropriate for their application.

Regardless, I agree that you should probably make gurobi optional since it isn't core to PySINDy and could cause headaches in some environments (e.g. when running on a distributed cluster).

[wesg52]

Yes I would want to do a once over on miosr.py. Some of the documentation is lazy at best. There is also some helper code for the paper that I would want to clean up.

All of the other code in the repo is for running and analyzing experiments, and can safely be ignored for the purposes of this integration.

[wesg52]

I submitted a PR with my changes to the MIOSR branch.

Regarding tests:

• I added a default sparsity which fixed the default argument tests (though not clear if that is actually the 'right' thing to do).
• Still failing anything that passes in threshold or max_iter (I am assuming the right thing to do is just remove miosr from those tests?)
• Also failing test_complexity (which wasn't obvious to me how to fix) and test_complexity_parameter[SR3-threshold] (which I assume is flakiness in SR3).
• For testing bad parameters, probably just want to check that the sparsities/alpha are positive.

The only outstanding design question is how best to enable users to have fine grained control over the gurobi model. Perhaps cleanest is just to force users to inherit from MIOSR and redefine _make_model, but this feels a little unsatisfying for making small tweaks (e.g. making the coefficients integer as well, which is really just a change of one argument).

Regardless, I think I am happy with the current state. Let me know if there is anything else I can help out with.

[Jacob-Stevens-Haas]

Thanks @wesg52! I'll need to read the paper & code more thoroughly in the next week to understand the details about parameters, but I'll get to this soon.

I only created the MIOSR branch to make it easier for you to fork & get started; now that you've forked it, feel free to PR into dynamicslab:master and let dynamicslab:miosr fall behind.

[akaptano]

This was finished and available now in PySINDy v1.7.2 onwards. Thanks to all!