Update 1.7.1 -> 2.1.0: typedefed enums as argument not longer working

[araisch]

With cppyy version 1.7.1 i wrapped many functions with pointer (=output) arguments and typedefed stuff. For example C++ function:

MLPI_API MLPIRESULT mlpiLogicGetTypeOfSymbol(const MLPIHANDLE connection, const WCHAR16 *symbol, MlpiLogicType *type, MlpiLogicType *subtype=0);

typedef enum MlpiLogicType
{
  MLPI_LOGIC_TYPE_BOOL                      =   0,  //!< 1 Byte (BOOL8)
  MLPI_LOGIC_TYPE_BIT                       =   1,  //!< Symbolic access unsupported (1 Bit)
<snipp>
  MLPI_LOGIC_TYPE_UNSUPPORTED               = 255   //!< Symbolic access unsupported
} MlpiLogicType;

In 1.7.1 it was running like that:

def MLPIGetSymbolType(self, symbol:str, layer:int) -> int:
        varType = ctypes.c_uint(0)
        subtype = ctypes.c_uint(0)
        ret = cppyy.gbl.mlpiLogicGetTypeOfSymbol(self.con.value, symbol, ctypes.pointer(varType), ctypes.pointer(subtype))
        if ret != 0x360000:
            raise Exception("MLPI Error: " + str(ret))
        if layer == 0:
            return varType.value
        else:
            return subtype.value

Pretty straight forward. But with 2.1.0 it drops:

TypeError: int ::mlpiLogicGetTypeOfSymbol(const MLPIHANDLE connection, const WCHAR16* symbol, MlpiLogicType* type, MlpiLogicType* subtype = 0) =>
    TypeError: could not convert argument 3 (could not convert argument to buffer or nullptr)

What do you mean by that? Am I not longer allowed to send arguments to a buffer? Sorry, I don't get it.

[araisch]

OK now i completely out of the game. If i add this testType = cppyy.gbl.MlpiLogicType(0) in the function above:

def MLPIGetSymbolType(self, symbol:str, layer:int) -> int:
        varType = ctypes.c_uint(0)
        subtype = ctypes.c_uint(0)
        testType = cppyy.gbl.MlpiLogicType(0)
        ret = cppyy.gbl.mlpiLogicGetTypeOfSymbol(self.con.value, symbol, ctypes.pointer(varType), ctypes.pointer(subtype))
        if ret != 0x360000:
            raise Exception("MLPI Error: " + str(ret))
        if layer == 0:
            return varType.value
        else:
            return subtype.value

Notice: this variable is not used anywhere. then it works. I have the deepest respect for what you are doing there, but my horizon is way lower. Feels like black magic to me. Maybe you find the time to enlighten me, how to solve it that it really feels like a solution? :)

[wlav]

I'm trying to create a self-contained reproducer, but it all works for me:

import cppyy, ctypes
  
cppyy.cppdef("""\
namespace enum_passing {

typedef enum LogicType {
  LOGIC_TYPE_BOOL                      =   0,
  LOGIC_TYPE_BIT                       =   1
} LogicType;

bool enum_by_pointer(LogicType* type) { *type = LOGIC_TYPE_BIT; return true; }

}""")

ns = cppyy.gbl.enum_passing

varType = ctypes.c_uint(0)
assert ns.enum_by_pointer(ctypes.pointer(varType))
assert varType.value == ns.LogicType.LOGIC_TYPE_BIT

My best guess, based on your 2nd comment, is that there is a Python error in creating the enum type that isn't properly cleared (and with that "ghost" variable, there is no error, hence no problem). Point being that buffer protocol can be implemented in various ways, so rather than checking for buffer types, it just attempts conversion, then responds to failure. In similar vein, it may be that this function works when calling it a second time.

Aside, is it certain that the enum type in an unsigned int? I.e., what happens if you use ctypes.c_int instead?

[araisch]

Aside, is it certain that the enum type in an unsigned int? I.e., what happens if you use ctypes.c_int instead?

TypeError: int ::mlpiLogicGetTypeOfSymbol(const MLPIHANDLE connection, const WCHAR16* symbol, MlpiLogicType* type, MlpiLogicType* subtype = 0) =>
    TypeError: could not convert argument 3 (could not convert argument to buffer or nullptr)

Seems so.. Same error then.

[wlav]

Is this the same enum as in #4? (I.e. the one that establishes that yes, indeed, the enum type is unsigned int?)

Actually, I just remembered ... cppyy-bound enum types have a __underlying data member that contains the name pf the actual underlying C++ type as well as a __ctype__ data member. I'm afraid I still need to update the documentation on that one ... it's quite useful. :)

Anyway, the point is that to find the ctypes type to use, you don't need to guess (at it's compiler and thus platform-dependent anyway), but simply do cppyy.gbl.MlpiLogicType.__ctype__(0). This may "fix" the problem in hiding it (like your example shows above), so it may not be a good end-all solution, but at least the value of cppyy.gbl.MlpiLogicType.__underlying can tell us whether cppyy and ctypes agree on the type of the enum?

[wlav]

And just to be sure __underlying was introduced in 1.8.3 (per the changelog :) ).

[araisch]

Using 2.1.0

testType = cppyy.gbl.MlpiLogicType(0)
print(cppyy.gbl.MlpiLogicType.__ctype__)
cppyy.gbl.MlpiLogicType.__ctype__(0)
print(cppyy.gbl.MlpiLogicType.__ctype__)

<bound method __ctype__ of <class 'MLPIWrapper.MlpiLogicType'>>

Traceback (most recent call last):
  File "/cmmlpipy/src/MLPIWrapper.py", line 278, in MLPIReadSymbol
    varType = self.MLPIGetSymbolType(symbol, 0)
  File "/cmmlpipy/src/MLPIWrapper.py", line 261, in MLPIGetSymbolType
    cppyy.gbl.MlpiLogicType.__ctype__(0)
TypeError: Can not find ctypes type for "internal_enum_type_t"

and

print(cppyy.gbl.MlpiLogicType.__underlying)

Traceback (most recent call last):
  File "/cmmlpipy/src/MLPIWrapper.py", line 278, in MLPIReadSymbol
    varType = self.MLPIGetSymbolType(symbol, 0)
  File "/cmmlpipy/src/MLPIWrapper.py", line 260, in MLPIGetSymbolType
    print(cppyy.gbl.MlpiLogicType.__underlying)
AttributeError: type object 'MlpiLogicType' has no attribute '_MLPIPy__underlying'

MLPIPy is the Class Name of the function MLPIGetSymbolType

[wlav]

The problem of not resolving the enum type properly may well have been fixed with: https://bitbucket.org/wlav/cppyy/issues/366/translation-alignment-issue-with-structs

Maybe that will make the original problem "go away."

I was hoping that by "forcing" the enum type to be internal_enum_type_t to reproduce the original error, but I could not. In fact, I can only pass ctypes.c_int through internal_enum_type_t (as it should).