optimus
optimus copied to clipboard
Move Yaml specs and their reader to client side
Description Currently the code for cli to read the yaml configuration related to job and resource are present along with server side code. We need to move these to client side, to establish the clear contract for input by users.
Acceptance Criteria
The client side code which defines user contracts as yaml should be moved to cli side of the code and not be used by the server side.
Out of Scope
- Changing of any api contracts
Tech Details Post analysis Capture the details of the work needed to finish the task.
- [ ] Simplify reading of different spec files
- [ ] Move reading of yaml spec to client, remove usage of them from server, eg local.Job
- [ ] Break away from common interface for Postgres and local store, move local store to client side
- [ ] Remove filesystem cache and implement file/folder reading
- [ ] Do not use server side models on client, convert yaml spec to proto contracts directly
Result of discussion about the approach on 2022-09-21:
- we expecte proto not to be changed
- ~split the bigquery specific structure, such as
BQTableMetadata
into smaller one~ (update based on discussion 2022-09-30: it will be part of this issue) - validation happens in server, meaning client is not expected to do such
- client only read, convert to proto directly, and send to the server
- the implementation addresses both job and resource