pdr-backend
pdr-backend copied to clipboard
[Lake][ohlcv] Improve error logging, such that problematic data can be logged/inspected.
trafficstars
Motivation
As a result of #657 it may be beneficial to have the "data being operated with" be logged at some capacity such that we can see the exact cause of these issues.
Rather than just logging an error during exception handling, provide more context w/ showing the data/dataframe or making sure this data is logged somewhere such that it be diagnosed.
Consider other lake consumers (gql) and steps (etl), and their error logging as it relates to the data/expected values they are working with.
DoD:
- [ ] Error/logging has been updated to provide problematic data, not just error message. In this case, ohlcv data/rows with a float-timestamp (likely a decimal rather than int) should have been accessible.