hermes icon indicating copy to clipboard operation
hermes copied to clipboard

Support different Hermes buffering modes

Open ChristopherHogan opened this issue 4 years ago • 0 comments

A Hermes application sets a global mode (Buffer Mode by default), and each Bucket or Put can override the mode via the Context.

Buffering Mode

The standard mode, where Blobs are buffered in the hierarchy and deleted when the system exits.

Persistent Mode

When Hermes shuts down, all buffered data and any metadata required to retrieve that data is persisted to a "final destination" as an HDF5 file. This is described in #119.

Direct I/O Mode

Blobs are not placed in the hierarchy, but are written directly to a "final destination," which is a file at a mount point specified by the user in the configuration file. Data is dumped as byte arrays in the order in which they are Put. Bucket level only -- no Put granularity.

  • Can "protect" data (fault tolerance)
  • Preserve hierarchy space

ChristopherHogan avatar Feb 10 '21 20:02 ChristopherHogan