pdr-backend icon indicating copy to clipboard operation
pdr-backend copied to clipboard

[Lake][Subgraph] Clean up subgraph objects, use pydantic table objects instead.

Open idiom-bytes opened this issue 1 year ago • 1 comments

Background / motivation

Subgraph objects (Prediction, Slot, Payout, Subscription) are unnecessary and redundant.

Subgraph fetching is part of data-integration, the flow should be:

  1. to validate it
  2. save it to CSV
  3. add to duckdb-lake

Perhaps, make sure that pydantic update is in place - #973

TODOs / DoD

  • [ ] Remove redundant data objects inside subgraph
  • [ ] Utilize pydantic objects for validating data coming from subgraph
  • [ ] Update tests or anywhere else as required

idiom-bytes avatar Apr 30 '24 15:04 idiom-bytes

@idiom-bytes I think this can be closed, right?

calina-c avatar May 21 '24 10:05 calina-c

Yes it can, closing...

idiom-bytes avatar May 27 '24 18:05 idiom-bytes