RFC: Tensorflow Core as a product

[sun51]

Tensorflow Core as a product RFC

Status	Proposed
Authors	Yanhua Sun ([email protected])
Sponsor	Rohan Jain ([email protected])
Updated	2020-11-13

Objective

This RFC proposes Tensorflow core as a standalone product and describes the high level components and the principles behind it.

[bhack]

Generally I think that this RFC will have some positive impacts if the SIGs leads will be involved in the process and not only in the RFC revision.

What I don't understand is if the scope of this RFC is to have a TF-core only repository in the near future maintained by a specific TF-core team. This Is related to the C++/C/Python API relationship. Just to make some examples:

• Keras Is a python only SiG but when It needs c++ ops are contributed in the TF monolitich repo as needed (e.g. image processing)

• TF.data, if It will be a separare SIG, It requires c++

• TF Addons and TFIO have custom ops when composabilty or tf.function is not enought to reach performance. But we know the portability ISSUES related to custom ops (https://github.com/tensorflow/custom-op/issues/53)

Are we going in a TF world where many SIGs will Need to maintain its own c++/c API to offer High level python API for all the cases not covered by python composabilty on TF-core or when the compiler stack (tf.function) is not efficient enought?

Also, are we sure that we can still minimize duplications across the SIGs ecosystem if the "centralized" core is going to have a minimal surface? I strongly belive that we will need a robust ecosystem review process with a smaller core proposal.

[mihaimaruseac]

~~There will be a design meeting for this on 15th of December at 10 am Pacific.~~

~~If you want to join, here is the Meet link~~

Edit: meeting has been postponed, see https://github.com/tensorflow/community/pull/331#issuecomment-744541408

[sun51]

There will be a design meeting for this on 15th of December at 10 am Pacific.

If you want to join, here is the Meet link

Sorry we need more time to figure out things better, the design review is cancelled for now, we will let you know if there is one later.

[sun51]

Thanks everyone for the discussion and comments, we will need more time to figure out the design, so we will hold this design RFC and have cancelled the design review for now. Will update later.

[bhack]

Thanks everyone for the discussion and comments, we will need more time to figure out the design, so we will hold this design RFC and have cancelled the design review for now. Will update later.

Do you mean https://github.com/tensorflow/community/pull/331#issuecomment-736889027 ?

[Craigacp]

Would this TF-core have a C API as well as a C++ one, or is that planned to live elsewhere?

[SidneyLann]

Hope most things do in c/c++

[ematejska]

@rohan100jain Are we still pursuing this RFC?

[bhack]

It would be really useful if we could revamp and approve this RFC.

[bhack]

/cc @learning-to-play

[bhack]

Do we have an opportunity to re-evalute/refresh this and all the useful feedback we had collected?

[learning-to-play]

@joker-eph Do you think there is an opportunity to incorporate the collected feedback into the current refactoring projects? See @bhack's comment above.