Add support for the HiGHS solvers via scipy

[mkoeppe]

Summary

scipy has started to vendor the HiGHS solvers. Providing a Pyomo solver based on this interface would give clean access to high-performance solvers, unencumbered by problematic licenses.

Rationale

Description

See https://github.com/scipy/scipy/issues/14455

Additional information

[bknueven]

I would be happy to take a pass at this in my spare time, once scipy/scipy#14455 is resolved. The HiGHS/scipy developers have created Python wrappers for HiGHS within scipy, which may be better for Pyomo to interface with as opposed to going through the linprog scipy interface.

[jajhall]

A direct interface to HiGHS from Pyomo is in the pipeline, and https://github.com/Pyomo/pyomo/tree/main/pyomo/solvers/plugins/solvers suggests that interfaces to solvers, rather than interfaces like scipy are preferred, particularly when re-solving LPs, or adding cuts to MIPs - which will be possible once HiGHS has call-backs.

The HiGHS team doesn't have infinite resources so, great a scipy is (and it's good that the scipy-HiGHS link for MIP is going ahead) we'd rather that the HiGHS were used directly from Pyomo

[mkoeppe]

A direct interface to HiGHS from Pyomo is in the pipeline

Will a future Pyomo version include a vendored copy of HiGHS then (in source distribution and/or wheels)?

[bknueven]

Will a future Pyomo version include a vendored copy of HiGHS then (in source distribution and/or wheels)?

I would very much doubt it. Pyomo does not currently distribute source or wheels for any other solver. In general, I point users to conda packages for various solvers, which exist for CBC, GLPK, and IPOPT, and probably others. Several commercial MIP solvers with their associated Python interfaces are also available on PyPI.

The only reason to leverage the scipy-HiGHs interface (or really, that HiGHs is distributed as part of scipy) is that it could be the most straightforward way for Pyomo users to obtain HiGHs. But if distribution via either conda or PyPI is already planned, it's a moot point.

[mkoeppe]

The only reason to leverage the scipy-HiGHs interface (or really, that HiGHs is distributed as part of scipy) is that it could be the most straightforward way for Pyomo users to obtain HiGHs.

Yes, exactly – this was the main point why I was excited about scipy shipping HiGHs.

But if distribution via either conda or PyPI is already planned, it's a moot point.

Not quite. The SciPy wheels do not provide HiGHs as a shared library; instead, HiGHS is shipped only as part of the Cython extension modules in scipy/optimize/_highs. For example:

$ otool -L scipy/optimize/_highs/_highs_wrapper.cpython-39-darwin.so 
scipy/optimize/_highs/_highs_wrapper.cpython-39-darwin.so:
	/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 400.9.4)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.250.1)

SciPy packaging on conda-forge could theoretically go through a HiGHs conda package which could then be reused by Pyomo packaging on conda-forge; but this is also not what is done: https://github.com/conda-forge/scipy-feedstock/blob/master/recipe/meta.yaml

[jajhall]

Thanks for flagging up Conda as a packaging system. The HiGHS team will look into it.

[michaelbynum]

PR #2245 adds a Pyomo interface to HiGHS through pyomo.contrib.appsi (https://pyomo.readthedocs.io/en/stable/library_reference/appsi/appsi.html). It uses a thin wrapper around the Highs c++ class, so the Highs model can be modified efficiently (e.g., for resolves). There is still work to do, but the basics are there. There are installation instructions in the docs in the PR. It can be used as

opt = appsi.solvers.Highs()
results = opt.solve(m)

There is a wrapper to get a traditional Pyomo solver interface as well:

opt = pyo.SolverFactory('appsi_highs')
results = opt.solve(m)

[davebiagioni]

This is awesome, thanks to all!