tfx-addons icon indicating copy to clipboard operation
tfx-addons copied to clipboard

Published 20 hours ago • verified publisher icon tensorflow

ExampleFilter

Open theadactyl opened this issue 3 years ago • 3 comments

Component to filter examples of a TF-record dataset using a user-defined predicate function

cc @sngahane

theadactyl avatar Apr 04 '21 01:04 theadactyl

Not up to date with TFXIO. Uses predicate to filter examples, which are represented as dicts. Needs to be refactored to make more robust. Similar to old Transform. Ryan to write proposal. cc @rclough

rcrowe-google avatar May 12 '21 18:05 rcrowe-google

Given the discussion around additional IO costs of making this a new component and the general consensus that this would be an generically useful addition for all ExampleGens, what would be the process for getting this considered into core TFX as part of ExampleGen instead of as an addon?

In the mean time we could continue on with this to get it usable.

pselden avatar May 12 '21 18:05 pselden

@deep-diver this sounds like something we have done something similar in our data drift project. Would you like to join forces here?

sayakpaul avatar Nov 24 '21 04:11 sayakpaul