itiyama

Idempotency for mostly append-only indexes/streams

5

**Background** Document id serves as a primary key in an Opensearch index. The distribution of the document across shards is based on the document id. Opensearch ensures idempotency on the...

Infer and cache date field format instead of re-parsing it for every document

2

The date field for the default format uses high CPU during parsing. A huge portion of date formatting time(close to 7.12% of CPU time in profiles) goes into parsing, which...

Parsing input formats beyond JSON...

1

JSON is, by far, the most popular input format for Opensearch. JSON is human readable, so it is pretty easy to test and develop with. Additionally, the Opensearch ecosystem is...

From [Lucene's](https://cwiki.apache.org/confluence/display/lucene/ImproveIndexingSpeed) documentation, **Re-use Document and Field instances** As of Lucene 2.3 there are new setValue(...) methods that allow you to change the value of a [Field](http://lucene.apache.org/java/docs/api/org/apache/lucene/document/Field.html). This allows you...

Modify read directory for files/blocks that are opened for merging.

1

Opensearch uses either MmapDirectory or NioFSDirectory for reading files, depending on the file type. This works well for files that are opened for search operation, where the entire file is...