greptimedb icon indicating copy to clipboard operation
greptimedb copied to clipboard

Remove `InsertBatch` in gRPC message

Open MichaelScofield opened this issue 2 years ago • 2 comments

What type of enhancement is this?

API improvement, Performance, Refactor, Tech debt reduction, User experience

What does the enhancement do?

before:

message InsertExpr {
  string schema_name = 1;
  string table_name = 2;

  message Values {
    repeated bytes values = 3; // which is actually the serialized value of InsertBatch
  }

  oneof expr {
    Values values = 4;
    string sql = 5;
  }

  uint32 region_number = 6;
  map<string, bytes> options = 7;
}

message InsertBatch {
  repeated Column columns = 3;
  uint32 row_count = 4;
}

after:

message InsertExpr {
  string schema_name = 1;
  string table_name = 2;

  // directly embed message here, no raw bytes
  repeated Column columns = 3;
  uint32 row_count = 4;

  uint32 region_number = 5;
  map<string, bytes> options = 6;
}

pros:

  1. get rid of "sql" in InsertExpr, which was introduced before as a temporary solution to insert data via frontend's grpc interface, because we cannot get table schema at that time
  2. much simpler to use in grpc cli tool, otherwise we have to serialize InsertBatch to bytes then to base64 manually, which is rather tedious
  3. no ser/de costs of InsertBatch on both grpc client and server
  4. a lot easier to write grpc related codes

cons:

  1. less flexible in InsertExpr, we cannot use another representation of insert values

first step to #490

Implementation challenges

No response

MichaelScofield avatar Nov 16 '22 09:11 MichaelScofield

after this is done, we should update doc

  • https://docs.greptime.com/user-guide/write-data/overview
  • https://docs.greptime.com/user-guide/query-data/overview

MichaelScofield avatar Nov 16 '22 10:11 MichaelScofield

You will also need to refactor common_insert::insert::build_create_expr_from_insertion's insert_batches argument to InsertExpr

v0y4g3r avatar Nov 18 '22 02:11 v0y4g3r