data-validation icon indicating copy to clipboard operation
data-validation copied to clipboard

`generate_statistics_from_csv` without blocking code execution?

Open santiagotoscanini opened this issue 4 years ago • 2 comments

Hi!

Is there a way to don't block the execution when calling the generate_statistics_from_csv?

Maybe return an Operation.

santiagotoscanini avatar Aug 19 '21 16:08 santiagotoscanini

What do you mean by not block the execution? Are you looking for it to "run in the background" or for a way to get the execution deferred? Note that generate_statistics_from_csv is a convinience method, an alternative is to write the beam pipeline manually if you'd like to tweak the pipeline options. Read the data with tfx_bsl.public.tfxio.CsvTFXIO, then call tfdv.GenerateStatistics followed by tfdv.WriteStatisticsToTFRecord.

zoyahav avatar Aug 25 '21 10:08 zoyahav

Hi!

Like many other Google APIs support, I look for a way to get the execution deferred and fetch the results with an operation name. Is there anything similar to this already implemented?

santiagotoscanini avatar Sep 01 '21 21:09 santiagotoscanini