api icon indicating copy to clipboard operation
api copied to clipboard

RHINENG-11927 - New API for uploading tsdb blocks

Open patilsuraj767 opened this issue 6 months ago • 8 comments

This PR adds new /api/v1/upload API where user can upload tsdb blocks packed in .tar.gz archive.

Below is the short summary of how upload API will work.

  1. User uploads the tsdb blocks packed in .tar.gz file. User can also add external_labels field in the multipart request. These external labels will be further added to all time series present in tsdb block.
    curl -v -F "external_labels={\"cluster_id\":\"123\"}" -F "file=@/tmp/tmp/test.tar.gz" -H "Authorization: Bearer ${TOKEN}"  
    http://localhost:8080/api/metrics/v1/test-oidc/api/v1/upload
    
  2. Once the request is received to the obs-api it checks the file size and later extract the tar archive.
  3. After extraction of tar, we read each tsdb blocks and creates the new blocks by adding the external_labels.
  4. later these new blocks are uploaded to the object storage.

patilsuraj767 avatar Aug 26 '24 09:08 patilsuraj767