Any hints on how to use the TCP API?

[RudolfVonKrugstein]

Hi,

The TCP API seems to be not documented (yet). Any hints, or simple example on how to use it? I would like to use onnxruntimer-server for serving silero VAD and using it via TCP api.

Thanks you!

[kibae]

Hi, @RudolfVonKrugstein

You're right, there's no documentation for the TCP API yet. I created it to work with the pg_onnx project, I didn't think anyone would use it. I'll try to explain it briefly and write some documentation after my vacation.

https://github.com/kibae/onnxruntime-server/blob/c2b2141652935bcf3aa5dbb200c2aa10e3d9b42a/src/transport/tcp/tcp_server.hpp#L21-L26 The structure of the protocol headers. The header contains the kind of operation you are requesting, the length of the entire request (JSON+POST), the length of the JSON data, and the length of the POST data (non-JSON data). The types of operations are defined here. https://github.com/kibae/onnxruntime-server/blob/c2b2141652935bcf3aa5dbb200c2aa10e3d9b42a/src/onnxruntime_server.hpp#L181-L187

After the header, you can send JSON data and after that POST data. Attach some of the E2E test code. https://github.com/kibae/onnxruntime-server/blob/c2b2141652935bcf3aa5dbb200c2aa10e3d9b42a/src/test/e2e/e2e_test_tcp_server.cpp#L16-L27

Have a great holiday season! 🥳

[RudolfVonKrugstein]

Thank you! I will try it after the holidays. And I will leave the issue open for you to decide if you want to close it now or when there is documentation.