ML icon indicating copy to clipboard operation
ML copied to clipboard

Missing callbacks related to datasets

Open khatchad opened this issue 2 years ago • 1 comments

Missing experimental_distribute_datasets_from_function

Called here: https://github.com/mead-ml/mead-baseline/blob/5d7632bb151c2d09501ebf49f36ba8c4204df4c8/mead/api_examples/pretrain_discrim_tf.py#L307

The callback function is dataset_test_fn, defined here: https://github.com/mead-ml/mead-baseline/blob/master/mead/api_examples/pretrain_discrim_tf.py#L301-L306

But that function doesn't show up in the call graph. The method reference should be: < PythonLoader, Lscript pretrain_discrim_tf.py/get_dataset, do()LRoot; >. However, we don't see it in the call graph nodes: https://gist.github.com/khatchad/bd48384f3561fdcc7c8c6e9141d93de5

Related to #92 and blocked on #89.

khatchad avatar Dec 04 '23 22:12 khatchad

We might be at the point where we can now fix this if we can treat distributed values as datasets.

khatchad avatar Mar 01 '24 18:03 khatchad