pdr-backend icon indicating copy to clipboard operation
pdr-backend copied to clipboard

[ETL][Lake] - Update existing code to query database by using PersistentDataStore.query_data

Open idiom-bytes opened this issue 1 year ago • 0 comments
trafficstars

Motivation

  • Review pds.query_data.
  • Consider not returning a polars lazy_frame but something else... perhaps a generic pointer where you can then do more with it.

DoD:

  • [ ] Updated existing logic that queries data from PDS/SQL, to get a better pointer to the data
  • [ ] When using polars/pandas, the usage should be stable
  • [ ] Tests are updated and passing

idiom-bytes avatar Mar 04 '24 22:03 idiom-bytes