Using Highs in cluster network optimizaion

[trevorb1]

Checklist

• [X] I am using the current master branch
• [X] I am running on an up-to-date pypsa-usa environment. Update via conda env update -f envs/environment.yaml

The Issue

When using highs in the cluster/simplify network scripts, an error is thrown by pyomo. If only highs is passed to the SolverFactory, the following error is thrown:

opt = po.SolverFactory("highs")
opt.solve(m)

Solver (asl) did not exit normally
  File "/home/trevor/master/pypsa-usa/workflow/scripts/cluster_network.py", line 247, in distribute_clusters
    results = opt.solve(m)
              ^^^^^^^^^^^^
  File "/home/trevor/master/pypsa-usa/workflow/scripts/cluster_network.py", line 317, in busmap_for_n_clusters
    n_clusters = distribute_clusters(
                 ^^^^^^^^^^^^^^^^^^^^
  File "/home/trevor/master/pypsa-usa/workflow/scripts/cluster_network.py", line 379, in clustering_for_n_clusters
    busmap = busmap_for_n_clusters(
             ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/trevor/master/pypsa-usa/workflow/scripts/simplify_network.py", line 278, in <module>
    clustering = clustering_for_n_clusters(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^
pyomo.common.errors.ApplicationError: Solver (asl) did not exit normally

If you pass the solver appsi_highs, as described by pyomo with the command pyomo help --solvers, the following error is raised:

opt = po.SolverFactory("appsi_highs")
if not opt.has_capability("quadratic_objective"):
...

Traceback (most recent call last):
  File "<string>", line 1, in <module>
AttributeError: 'LegacySolver' object has no attribute 'has_capability'

Steps To Reproduce

1. Change solver configuration to the following:

  solver:
    name: highs
    options: highs-default

2. Run the workflow

Expected Behavior

The clustering optimization should run via ipopt if highs is selected for the quadratic optimization

https://github.com/PyPSA/pypsa-usa/blob/a05664935c3d0828b9dae46d64fcd4fca137d177/workflow/scripts/cluster_network.py#L186-L252

Error Message

No response

Anything else?

The pypsa-eur team updated their clustering to use linopy; would be nice to follow their implementation and allow us to drop the pyomo dependency

https://github.com/PyPSA/pypsa-eur/blob/51f8c2935ae3f80b15ad0ab71708ad006ad49da9/scripts/cluster_network.py#L172-L230

[trevorb1]

Thanks for the patch, @ktehranchi! I agree with pushing this to master. You okay if I reopen this though, as the patch doesnt address the underlying problem of highs not being used for the cluster network?

[ktehranchi]

Agreed- i also had an issue with pyomo on my cluster runs this week... it wasn't reading my gurobi license correctly, so i might take this on and replace pyomo with linopy soon