optimus icon indicating copy to clipboard operation
optimus copied to clipboard

Move Yaml specs and their reader to client side

Open sbchaos opened this issue 2 years ago • 1 comments

Description Currently the code for cli to read the yaml configuration related to job and resource are present along with server side code. We need to move these to client side, to establish the clear contract for input by users.

Acceptance Criteria

The client side code which defines user contracts as yaml should be moved to cli side of the code and not be used by the server side.

Out of Scope

  1. Changing of any api contracts

Tech Details Post analysis Capture the details of the work needed to finish the task.

  • [ ] Simplify reading of different spec files
  • [ ] Move reading of yaml spec to client, remove usage of them from server, eg local.Job
  • [ ] Break away from common interface for Postgres and local store, move local store to client side
  • [ ] Remove filesystem cache and implement file/folder reading
  • [ ] Do not use server side models on client, convert yaml spec to proto contracts directly

sbchaos avatar Sep 08 '22 14:09 sbchaos

Result of discussion about the approach on 2022-09-21:

  • we expecte proto not to be changed
  • ~split the bigquery specific structure, such as BQTableMetadata into smaller one~ (update based on discussion 2022-09-30: it will be part of this issue)
  • validation happens in server, meaning client is not expected to do such
  • client only read, convert to proto directly, and send to the server
  • the implementation addresses both job and resource

irainia avatar Sep 22 '22 03:09 irainia