delira
delira copied to clipboard
[FeatureRequest] Prepare for tf 2.0
Description Prepare tf and tf-eager backends for migration to tf 2.0
Proposal
-
deprecate graph mode and remove it when migrating to tf 2.0, as speed loss can and should be minimize with the eventual @tf.function decorator that does jit compilation + optimizaiton.
-
Refactor tf-eager backend to utilize functions stored inside tf.compat.v2, that are guaranteed to be compatible with tf 2.0. Especially use optimizers, variables, losses and metrics stored there.
Additional context Design changes for tf 2.0 and their discussion can be read up on in the rfcs
@justusschock you want this to be in the PR #102? Or should I make a new one
I would prefer a new one based on this PR, since we should merge this separately (to first have an intermediate release supporting the TF1.14 Eager and Graph Mode).
Currently the eager backend is build upon tf.keras.layer. Since TF 2.0 strengthens the usage of tf.keras, I would like to keep it that way, what do you think?
I don't know if we shoud really deprecate the graph mode completely, since it will still be faster than jit-compilation AFAIK.
Reminder: Check if inheritance from tf.Module is better than inheritance from tf.keras.Layer [see #102 ]
I think, we should wait with adressing this until #221 is merged, since this should make it easier to derive trainers