Framework Ops vs Raw Ops

[JimClarke5]

In Python TensorFlow, there are some OPs defined in the Python Layer, and some defined in the C-api layer. I have been tasked to see how Java TensorFlow might want to handle this.

I have run some experiments with creating a FrameworkOperatorProcessor class in tensorflow-flow-generator and a couple of architectures present themselves. This class is basically a copy of OperatorProcessor with some tweaks.

The approaches seem to dictate generating a new class in tensorflow-framework, that I named FOps for now.

1. The first approach is to have FOps subclass org.tensorflow.op.Ops, that is generated in tensorflow-core-api. However, this leads to potential problems with name clashes with the methods and groups already in Ops. A prime example of this are the NN classes we added for Nn and NnRaw (SoftmaxCrossEntropyWithLogits<T> softmaxCrossEntropyWithLogits() has the same signature in both generated classes.) This option requires changing Ops from a final class to non-final so that it can be inherited.

2. A second approach is to use the delegate pattern, and have FOps hold an internal reference to Ops, and you could call methods on each as required. For example,

FOps ftf = FOps.create(graph);
ftf.math.tensordot(); // framework op
ftf.getOps().math.mul();  // raw op

3. Keep both totally separate from each other. This may potentially allow reuse of the existing OperatorProcessor. It may be more cumbersome to the programmer user.

4. Another option, that I haven't thought of yet.

I welcome thoughts on this.

[altavir]

Just got the same confusion with tensordot. It obviously is a basic operation.

[JimClarke5]

In Python TF, tf.tensordot is written in Python and calls various raw ops. Also,iIn TF Python, they moved the raw ops to the raw module, but the higher level ops are either at the tf level, or in modules like tf.math. There are also several aliases for certain ops.

I placed these ops in Framework, as there was a desire to keep the core ops to those that directly call the c api.

Also in the Python code, ops like tensordot are not strictly assigned to sub modules like math, even though the Python code resides there.

[altavir]

Dot is one of the most time-consuming operations. So it will require some work to do it properly. I am experimenting with KMath wrapper for tensorflow lazy operations, so I need it to implement the API.

Also, there is no general registry of operations which makes it a little bit tricky to understand how to do different things only from operation names (I am not familiar with the Python API).