horaedb icon indicating copy to clipboard operation
horaedb copied to clipboard

feat: Support remote engine server with flatbuffer write

Open zealchen opened this issue 1 year ago • 0 comments

Rationale

Our benchmarks show flatbuffers has the best perf, thus we replace the remote server write operation from protobuf to flatbuffers protocol.

Detailed Changes

  1. Add a new flatbuffer(remote_engine.fbs) protocol schema.
  2. Implement RemoteEngineFbService trait for RemoteEngineServiceImpl as the new flatbuffer-based service.
  3. Add Client/Server side implementation for flatbuffer write.

Test Plan

Manual test as follows:

  1. run horaemeta-server and two hoaredb-server on my Mac(M1) local machine.
  2. create a partition table:
curl --location --request POST 'http://127.0.0.1:5440/sql' \
-d '
CREATE TABLE `demo`(
    `name`string TAG,
    `id` int TAG,
    `value` double NOT NULL,
    `t` timestamp NOT NULL,
    TIMESTAMP KEY(t)
    ) PARTITION BY KEY(name) PARTITIONS 2 ENGINE = Analytic
'
  1. send write request:
curl --location --request POST 'http://127.0.0.1:15440/sql' \
-d '
INSERT INTO demo (t, name, value, id)
    VALUES (1702224000000, "horaedb", 100, 1)
'

zealchen avatar Jun 29 '24 23:06 zealchen