Required to set forwarding pipeline before using the dataplane for streaming packets

[matthewtlam]

I was wondering if it is necessary to create and set the forwarding pipeline in BMv2 before creating a dataplane stub for packet streaming?

For context, I wish to create the dataplane stub so that I can stream packets on the dataplane, but I would like to do this before setting the forwarding pipeline. I noticed that if I do not create p4runtime stubs and set the forwarding pipeline before creating the dataplane stub and stream, then I cannot read from the dataplane stream as it is closed. I was curious if this is the intended behavior of BMv2?

cc @smolkaj

[antoninbas]

Based on the code, it seems that it is the intended behavior: https://github.com/p4lang/behavioral-model/blob/c74c53661778cc564b7f8e1c1197241319516809/targets/simple_switch_grpc/switch_runner.cpp#L114-L115

If you cannot order things "correctly", you could always start simple_switch_grpc with a placeholder pipeline that drops all packets (or any other default behavior you prefer).

I would also be open to a PR that modifies the behavior so that the PacketStream implementation only returns an error if a packet is sent while there is no forwarding pipeline (as opposed to closing the stream right away which is the current behavior). I don't really see any downside to that.

I can't recall if other gRPC implementations (e.g. in Golang) have the same behavior as the C++ one, where the client only gets that error after the first call to Read.

[smolkaj]

Thanks, Antonin!

Providing some additional context on why opening a dataplane stream before pushing forwarding pipeline config is useful, in case that wasn't obvious.

We are working in a scenario where the 2 BMv2 grpc services (dataplane, P4RT) are used by different modules that we don't want to require to be synchronized. E.g., it makes sense for the dataplane sevice to be used by a "topology module" that implements connectivity between different switches, and for the P4RT service to be used by a "SDN controller module". In a physical network, the physical topology is oblivious to the SDN controller, and we want the same thing to be true in software.

cc @jonathan-dilorenzo

[smolkaj]

you could always start simple_switch_grpc with a placeholder pipeline that drops all packets (or any other default behavior you prefer)

Thanks for the suggestion. With this approach, we can decouple the topology module from the SDN controller module after this initialization step. But it has the downside that the SDN controller module will see the switch as already being configured, so the solution may require some special logic/awareness in the SDN controller module.

I would also be open to a PR that modifies the behavior so that the PacketStream implementation only returns an error if a packet is sent while there is no forwarding pipeline (as opposed to closing the stream right away which is the current behavior). I don't really see any downside to that.

Agreed. One nuance is that this will require terminating the stream in the error case, because AFAIK, that is the only way to communicate an error through the grpc bidirectional streaming API. I don't think there is a big problem with that.

[antoninbas]

But it has the downside that the SDN controller module will see the switch as already being configured

Actually, I think that if you are providing a bmv2 JSON file when starting the simple_switch_grpc process, as a command-line argument, from a P4Runtime perspective it will show as an "unconfigured" switch. This can either be considered as a bug or as a feature depending on what your needs are :)

[matthewtlam]

How would I provide a bmv2 JSON file to the simple_switch_grpc process?

Would it just be removing the --no-p4 flag and then adding a file path like simple_switch_grpc <JSON FILE> \ -i <PORT1>@<IFACE1> -i <PORT2>@<IFACE2> <more ports> \ -- --grpc-server-addr <IP>:<TCP PORT> --cpu-port <CPU_PORT>

[antoninbas]

Yes

[matthewtlam]

I tried to run simple_switch_grpc process and passed a BMv2 JSON file to it. Afterwards, when trying to create a P4runtime stub, I end up timing out. I was wondering if you had any idea why this might be happening?

[antoninbas]

The simple_switch_grpc logs (when using --log-console) should tell you if the JSON was loaded successfully. As far as I know, providing a JSON file this way has no impact on whether the P4Runtime server is started.

[matthewtlam]

Thank you Antonin! I implemented the workaround and it works as you mentioned. The dataplane stub can be retrieved when I pushed a barebones BMv2 JSON config in the command-line args

[smolkaj]

Thank you for the platinum support, @antoninbas, highly appreciated! :)