ConverJens

@jccarles There are two ways to do this, both of them has issues. 1. Package your code into a whl or similar and pass that as an extra package to...

@jccarles Actually, I created tfx python custom components. They are very similar to kubeflow components: https://www.tensorflow.org/tfx/guide/custom_function_component To make this work I actually created a second pipeline which spawned the flink...

@jccarles Solution 1 has the upside (if it works) that you only need to deploy one Flink cluster and that can be used for all jobs. This is better if...

@lukesolo This is because your tensorflow_io import only exists in your pipeline definition file, not in the ExampleGen component which is run in another process or container. This is fixed...

@jiyongjung0 The build is failing on building some dependency. Could you have a look? This does not happen for me when I build the TFX image and whls locally.

@jiyongjung0 I rebased agains master but it seems like the issue remained.

Hi @jiyongjung0, I've occupied by other stuff will try to make some progress on this PR this week if I can manage. Will let you know.

@jiyongjung0 The build fails on pre-existing proto lint failures now.

@jiyongjung0 @kennethyang404 I have created a unit test for the beam placeholder logic but it currently fails with the very unhelpful message: "not a cmessage" Any help understanding why this...

@jiyongjung0 It was, I used the wrong placeholder type. As of now, if I only use placeholders (no beam args as strings), the compilation succeeds but the component fails upon...