ustore icon indicating copy to clipboard operation
ustore copied to clipboard

Supporting in-document IDs and joined doc imports/exports

Open ashvardanian opened this issue 3 years ago • 0 comments

Benefits

Java, GoLang and many other bindings will receive "upsert" functionality with just a single char const * argument. Similarly, streaming exports can emplace ID into the packed document, to simplify post-processing for user. This form is compatible with Mongo DB and Elastic stack, which are behind in terms of Apache Arrow adoption.

Changes

  1. If no docs_count is set:
    • if the format is JSON - we count the newlines.
    • we need to have at least the first length variable set.
  2. Every input document is checked to contain an integer-castable top-level _id field.

ashvardanian avatar Sep 06 '22 21:09 ashvardanian