tfx-addons icon indicating copy to clipboard operation
tfx-addons copied to clipboard

TFX MLMD Client library

Open theadactyl opened this issue 3 years ago • 5 comments

Currently it's hard to query MLMD as it requires a lot of information on how TFX structures data internally in MLMD. There's several examples (including Airflow TFX example or Model Cards) that have repetitive functions just because they are not included in the core library. We can simplify MLMD querying by providing some of those functions ourselves.

cc @casassg

theadactyl avatar Apr 04 '21 01:04 theadactyl

Some small previous example from Twitter: https://github.com/tensorflow/tfx/pull/2415

casassg avatar Apr 14 '21 18:04 casassg

Can be used as a starting point for MLMD client library: https://github.com/google/nitroml/tree/master/nitroml/analytics

BACtaki avatar Apr 14 '21 18:04 BACtaki

rcrowe-google avatar Apr 14 '21 18:04 rcrowe-google

  • Easily query lineage, what features generated what model.

casassg avatar Apr 14 '21 18:04 casassg

Another similar example: https://github.com/zenml-io/zenml/blob/main/src/zenml/repository.py

casassg avatar Apr 27 '22 16:04 casassg