NL interface doesn't support python functions in `ExternalFunction`

[wangyexiang]

Summary

Steps to reproduce the issue

My code is:

# --*-coding:utf-8-*-
from pyomo.environ import *

m = ConcreteModel('test')
m.x = Var(bounds=(-2000, 2000))


def myfun(x):
    return x**2


m.fun = ExternalFunction(function=myfun)
m.obj = Objective(expr=m.fun(m.x))

opt = SolverFactory('ipopt')
results = opt.solve(m)

m.display()
...

Error Message

pyomo.common.errors.ApplicationError: Solver (ipopt) did not exit normally

But, this code works well:

# --*-coding:utf-8-*-
from pyomo.environ import *

m = ConcreteModel('test')
m.x = Var(bounds=(-2000, 2000))


def myfun(m):
    return m.x**2

m.obj = Objective(rule=myfun)
opt = SolverFactory('ipopt')
results = opt.solve(m)

m.display()

My another question is: How to use 'ExternalFunction' to solve a black box function of python?

Information on your system

Pyomo version: 6.0.1 Python version: 3.8 Operating system: win10 How Pyomo was installed (PyPI, conda, source): pip Solver (if applicable): ipopt

Additional information

[jsiirola]

This is tricky. The standalone "ipopt" solver is compiled against the ASL NL interface, which only supports compiled external functions adhering to the ASL specification. The challenge with native Python functions is that the ipopt solver runs as an independent subprocess. Getting it to be able to make calls back into the Python process that invoked it would either require building a custom ipopt executable with python bindings or else building a generic ASL external function library that could communicate with the current Python process through something like sockets (we prototyped this years ago, but it was very fragile and we never had a chance to polish / finish that code up).

There is some "alpha" functionality in PyNumero for an ExternalGreyBoxModel. This supports external functions implemented in Python; however, it only works with the cyipopt solver (ipopt python bindings).