OpenBBTerminal icon indicating copy to clipboard operation
OpenBBTerminal copied to clipboard

feature/units-of-measurement: Insert metadata for frontend handling of percent values.

Open deeleeramone opened this issue 5 months ago • 2 comments

This is an oversized PR that touches a lot of files and is a work-in-progress.

The end result would be a standard format to represent all percent values returned by the fetchers. Once all current providers conform to this, maintaining going forward would be pretty simple and can be expanded to include other unit types or metadata.

The overall purpose is to:

  • Identify all fields where the value represents a % and insert metadata to communicate this to frontend components.
    • json_schema_extra={"x-unit_measurement": "percent", "x-frontend_multiply": 100}

[X] Standard Models [X] Provider Models

  • [X] FMP
  • [X] FRED
  • [X] Intrinio
  • [X] Federal Reserve
  • [X] Benzinga - None
  • [X] Alpha Vantage - None
  • [X] Biztoc - None
  • [X] Cboe
  • [X] ECB
  • [X] Finra
  • [X] Government US
  • [X] Finviz
  • [X] Nasdaq
  • [X] OECD
  • [X] Polygon - None
  • [X] SEC
  • [X] Seeking Alpha - None
  • [X] Stockgrid
  • [X] Tiingo
  • [X] Tradingeconomics - None
  • [X] TMX
  • [X] WSJ
  • [X] YFinance

This inserts the metadata into openapi.json at each appropriate field.

  • Verify that any data returned to these fields actually conforms to this description.
  • Additionally, missing model definitions have been added and field names are standardized between providers wherever possible.

There is no automated process to do this, so don't take my word for it. Some places may require a validator on the provider model, other places are better conformed in the transform_data stage. Overall, it cannot account for 100% of all instances, dynamic fields and types need case-by-case decisions for potential exceptions to the general rule of representing % as a calculation input-ready decimal. Comprehensive reviews are absolutely required here.

  • [X] FMP
  • [X] FRED
  • [X] Intrinio
  • [X] Federal Reserve
  • [X] Benzinga - None
  • [X] Alpha Vantage - None
  • [X] Biztoc - None
  • [X] Cboe
  • [X] ECB
  • [X] Finra
  • [X] Government US
  • [X] Finviz
  • [X] Nasdaq
  • [X] OECD
  • [X] Polygon - None
  • [X] SEC
  • [X] Seeking Alpha - None
  • [X] Stockgrid
  • [X] Tiingo
  • [X] TMX
  • [X] Tradingeconomics - None
  • [X] WSJ
  • [X] YFinance

What this does not cover:

  • Functions where the data output can be dynamically transformed with query parameters
  • Functions where the units returned varies - i.e, FRED Series IDs, Economic Calendar.
  • Functions that are not fetching data - i.e, TA/QA/Econometrics

deeleeramone avatar Jan 18 '24 20:01 deeleeramone