kube icon indicating copy to clipboard operation
kube copied to clipboard

Raw store

Open SOF3 opened this issue 1 year ago • 3 comments

Motivation

Described in #1389. To summarize, RawJson has a smaller memory footprint compared to DynamicObject, only requiring 48 bytes of overhead (+ one JSON buffer allocation on heap) for each object for the smallest case.

Solution

Introduce a new raw_json module:

  • The type RawJson is a new ToObjectRef implementation that can be used in reflectors similar to how DynamicObject is used, with efficient access to the object ref fields required for reflector (i.e. namespace, name, resource version and uid)
  • The utility type StrOffset acts like a Range<usize>, providing utilities to convert from/to a substring &str in a JSON buffer
  • The Extra trait provides the ability to cache additional data (offsets or other lightweight data) in a RawJson
  • Extra is implemented by () to indicate that no raw data is requested.

SOF3 avatar Feb 11 '24 04:02 SOF3

Codecov Report

Attention: Patch coverage is 54.21687% with 38 lines in your changes are missing coverage. Please review.

Project coverage is 74.7%. Comparing base (22cb4a4) to head (f0f322d).

Additional details and impacted files
@@           Coverage Diff           @@
##            main   #1406     +/-   ##
=======================================
- Coverage   74.9%   74.7%   -0.2%     
=======================================
  Files         78      79      +1     
  Lines       6801    6884     +83     
=======================================
+ Hits        5091    5136     +45     
- Misses      1710    1748     +38     
Files Coverage Δ
kube-runtime/src/raw_json.rs 54.3% <54.3%> (ø)

codecov[bot] avatar Feb 12 '24 15:02 codecov[bot]

Further work is necessary to refactor kube::Api and/or runtime::watcher such that we can actually create a watcher that uses RawJson only.

SOF3 avatar May 08 '24 14:05 SOF3

AFAIKT there are at least 3 trait bounds we need to think about to make this viable (if we want to integrate it with watcher and Api as is):

  1. watcher' s trait bound on Resource
  2. Api bound on Resource
  3. For Api method ::list and ::watch, the DeserializeOwned bound

The first should be reduced to a trait bound on Lookup pretty easily (i think) without being particularly breaking. Api's Resource bound should only affect constructors, so if can make a new constructor for json usage we should be able to side-step this.

The last one, I am not sure about though.

clux avatar May 09 '24 11:05 clux