Calling Python functions from HPy

[hodgestar]

We need to pick some APIs to provide for calling Python functions from HPy.

The existing C API has a large zoo of methods for making functions calls. The zoo is documented at https://docs.python.org/3/c-api/call.html#object-calling-api.

Under the hood there are two main calling conventions -- tp_call and vectorcall. I propose that we largely ignore these and instead focus on what fits well with the Python language syntax and what is convenient to call from C.

Most Pythonic

Of the zoo, PyObject_Call most clearly matches a generic f(*args, **kw) in Python, and I propose that we adapt is as follows for HPy:

/* Call a callable Python object, with arguments given by the tuple args, and named arguments
    given by the dictionary kwargs.

    ctx: The HPy execution context.
    args: May be HPy_NULL if no positional arguments need to be passed. Otherwise it should be
             a handle to a tuple.
    kwargs: May be HPy_NULL if no keywords arguments need to be passed. Otherwise it should be
                 a handle to a dictionary of keyword arguments.

    Return the result of the call on success, or raise an exception and return HPy_NULL on failure.

    This is the equivalent of the Python expression: callable(*args, **kwargs).
 */
HPy HPy_Call(HPyContext ctx, HPy callable, HPy args, HPy kwargs);    

Note that this differs from PyObject_Call in that args may be NULL. I'm not sure why this was a requirement in the existing API.

A some point in the future we might want to implement Py_BuildValue to allow tuples of C values to be constructed easily (and maybe even something similar for easily constructing dictionaries from C values).

Most C-like (Most Scenic? :)

PyObject_VectorcallDict closely matches the signature we chose for HPyFunc_KEYWORDS methods, but while this is convenient for receiving arguments from Python, I'm not convinced it's a great fit for calling Python functions from C because one has to construct a dictionary of keyword arguments.

PyObject_Vectorcall takes only the names of the keyword arguments (as a tuple), which seems slightly more convenient.

All of the vectorcall methods have the strange behaviour that nargs also functions as a flag selecting whether args[-1] (or args[0] for PyObject_VectorcallMethod) may temporarily be overwritten by the called function. I propose that we ensure we do NOT copy this behaviour.

The best suggestion I have at the moment for a convenient C-like calling convention is:

/* Call a callable Python object with positional and keyword arguments given by a C arrays.

    ctx: The HPy execution context.
    args: An array of positional arguments. May be NULL if there are no positional arguments.
    nargs; The number of positional arguments.
    kwnames: An array of keyword argument names given as UTF8 strings. May be NULL if there
                     are no keyword arguments.
    kwargs: An array of keyword argument values. May be NULL if there are no keyword arguments.
    nkwargs: The number of keyword arguments.

    Return the result of the call on success, or raise an exception and return HPy_NULL on failure.

    This is the equivalent of the Python expression: callable(*args, **kwargs).
 */
HPy HPy_CallArray(HPyContext ctx, HPy *args, HPy_ssize_t nargs, const char **kwnames, HPy *kwargs, HPy_ssize_t nkwargs);

I'm uncertain whether this is to big a departure from existing APIs or whether there are better options.

Other considerations

• We should also implement HPyCallable_Check at the same time.
• We'll probably want to implement HPy_CallMethod and HPy_CallMethodArray, but I propose that we decide on these first and then ensure those match later.