TileDB icon indicating copy to clipboard operation
TileDB copied to clipboard

RFC Allow writing subset of attributes at a time

Open ihnorton opened this issue 4 years ago • 1 comments

  • allow writing only a subset of attributes at a time
  • provide API to set the exact fragment URI for write so that we can continue writing into an existing fragment folder
  • provide API to "unset" a buffer to allow setting the next (different) attribute buffer
  • provide API to signify that a query is a "continuation" write which will avoid various checks and preserve global write state until finalization
  • move several variables to class level for reuse to allow appending into fragment metadata object; avoid recalculating cell position across runs; also avoid re-checking duplicates

There may be an implicit requirement that attributes are set in schema order (that is how I did it in all cases, not sure if this is required for correct fragment metadata).


TYPE: FEATURE DESC: Allow writing subset of attributes at a time

ihnorton avatar Dec 22 '21 03:12 ihnorton

This is very WIP. I've only used for ~5 small writes, which were read back correctly. Putting up for comment to see if this approach is feasible, and so I don't lose track of the branch.

(I believe this would fix #-1162)

ihnorton avatar Dec 22 '21 03:12 ihnorton

Closing as stale, deferred.

ihnorton avatar Aug 23 '22 18:08 ihnorton