Support for various metadata store backends [need feedback and review]
In current implementation, the ML metadata (MLMD) metadata store was tightly coupled with CMF. Our experiments and standalone proof-of-concept implementations have demonstrated that it is possible to use other platforms, such as MLflow to store pipeline metadata (though, not as efficiently as with MLMD store, but in certain cases when a team or organization has already setup the ML metadata management service, they can start tracking their pipeline metadata using that existing service).
The idea is relatively straightforward - take MLMD related collection of functions and wrap them into a class (MlmdStore). Then functions that are used by Cmf become public interface for MlmdStore, and respectively, become API that Cmf requires from arbitrary metadata backend in the future.
This commit moves existing MLMD functionality into cmflib.metadata.mlmd_store module.
At this point, the implementation is merely a draft for the proposed idea. We are collecting feedback and suggestions.
This PR should also address CMF running in the debug mode, where the backend can be a console , and we do a dry run and no hard changes like commits and branches .