pycapnp

pycapnp copied to clipboard

Is it possible to implement a server for a generic Capnproto interface using pycapnp?

I suspect that designating the concrete type parameters is not currently possible, but I'd settle for an implementation that could accept AnyPointer as the parameters.

Hmm, I'm not sure.

Do you have an example of what you have in mind? I guess in theory a generic class/function could be written and just have everything a kwargs. But I'll have to look into it. Internally everything is generic, but I suspect the fields and methods use some sort of internal name lookup that's generated when compiling the schema. I believe that the servers inherit from _DynamicCapabilityServer (https://github.com/capnproto/pycapnp/blob/master/capnp/lib/capnp.pyx#L2091) and right now an exception is thrown if the attribute isn't found. Maybe some sort of callback can be added that will be called before an exception is thrown?

I was hoping that, given interface: interface Foo(X) {} then the Python implementation class MyFoo(Foo): pass would be sufficient to act as a capability of type Foo(AnyPointer).

I don't think that's true right now? At least, I get errors of the form:

"failed: expected capability.getSchema(),extends(interfaceType); Value type mismatch."

I might be missing something, and there already is a generated Python class that can act as a Foo(AnyPointer) capability?

Can you send me a basic example schema (just so I'm understanding things correctly; I'm not the original author so I've been trying to piece together what's going on)?

FWIW, I don't think pycapnp was ever explicitly updated to support generics. But, they mostly work by virtue of the way the C++ dynamic API was updated to support generics. For example, if you have a regular struct in pycapnp, and you get a field, and that field's type is a specialized generic type, the C++ dynamic API is automatically going to track that so pycapnp never actually has to be aware, because in this case the type name is never actually explicitly referenced on the Python side.

But if you want to refer to a specialization of Foo directly in Python, I guess you currently can'd do that.

I'm a little surprised, though. I would have expected that, by default, Foo would refer to the unspecialized generic, in which all type parameters are treated as AnyPointer. But @vaci is saying that's not what's happening? @vaci, are you sure about that? The error you reported looks to me like the error you'd get if you successfully created a Foo(AnyPointer), but then tried to use it somewhere a specific specialization of Foo was needed. Foo(AnyPointer) is not considered compatible with Foo(SomeSpecificType), so this is an error.

@vaci it would probably help if you provided a self-contained test program for @haata to play with.

Here's a short test program to demonstrate what I'm seeing:

import capnp
import sys
import tempfile

with tempfile.NamedTemporaryFile() as tmp:
    tmp.write('''
@0x89a3d41c2e94cac8;
interface Foo(Y) {}
interface Bar { func @0 (arg: Foo(AnyPointer)); }
'''.encode('utf-8'))
    tmp.flush()
    schema = capnp.load(tmp.name)

endpoint = "localhost:4646"

class Foo(schema.Foo.Server): pass

class Bar(schema.Bar.Server):
    def func_context(self, ctx, **kwargs):
        pass

if sys.argv[1] == 'client':
    client = capnp.TwoPartyClient(endpoint)
    bar = client.bootstrap().cast_as(schema.Bar)
    foo = Foo()
    req = bar.func_request()
    req.arg = foo
    req.send().wait()
else:
    server = capnp.TwoPartyServer(endpoint, bootstrap=Bar())
    server.run_forever()

I cannot send Foo() as the "arg" parameter:

Traceback (most recent call last):
  File "foo.py", line 35, in <module>
    req.arg = foo
  File "capnp/lib/capnp.pyx", line 1274, in capnp.lib.capnp._DynamicStructBuilder.__setattr__
capnp.lib.capnp.KjException: capnp/dynamic.c++:639: failed: expected capability.getSchema().extends(interfaceType); Value type mismatch.

Hello! I just stumbled over the same problem. Can I expect that anytime soon it would be possible to implement generic interfaces with pycapnp? If not I probably should for the time being change my schemas to not use generics at all. For prototyping pycapnp is pretty convenient and I use it currently all the time. Besides that, @haata, thanks a lot for maintaining pycapnp currently. Highly appreciated. 👍

Sorry it's taking me so long to look at this. 2020 has been quite a year... I might be able to look at this some time in September but I'm not sure at this point (especially given the fire situation here in California).

Thank you for the feedback. Don't worry, I can work around that currently. Just wanted to know if it is in the pipeline or not. Thanks for your efforts. 👍 :-)

Not sure if this is relevant or should be its own issue, but using a generic interface is also a bit inconvenient.

I have an interface like this

interface Simulator(Cmd) {
    loadFiles @0 (files :List(File)) -> (commands :Cmd);
}
interface Run {
    run @0 (vectors :List(Text)) -> (result :Result);
}

and on the python side I have to do something like the following:

sim = capnp.TwoPartyClient('localhost:5923').bootstrap().cast_as(api.Simulator)
res = sim.loadFiles([{"name": "bar.sp", "contents": ckt}]).wait()

runner = res.commands.as_interface(api.Run)

If there were support for generics, maybe you could do something like cast_as(api.Simulator(api.Run)) and you would not need to manually cast the return value.

A solution I found if you have only a few non-generic implementations, you can do the following to make a non-generic implementation.

interface Xyce extends(Simulator(Run)) { }