delira icon indicating copy to clipboard operation
delira copied to clipboard

[FeatureRequest] Prepare for tf 2.0

Open ORippler opened this issue 6 years ago • 4 comments

Description Prepare tf and tf-eager backends for migration to tf 2.0

Proposal

  1. deprecate graph mode and remove it when migrating to tf 2.0, as speed loss can and should be minimize with the eventual @tf.function decorator that does jit compilation + optimizaiton.

  2. Refactor tf-eager backend to utilize functions stored inside tf.compat.v2, that are guaranteed to be compatible with tf 2.0. Especially use optimizers, variables, losses and metrics stored there.

Additional context Design changes for tf 2.0 and their discussion can be read up on in the rfcs

@justusschock you want this to be in the PR #102? Or should I make a new one

ORippler avatar Jul 24 '19 08:07 ORippler

I would prefer a new one based on this PR, since we should merge this separately (to first have an intermediate release supporting the TF1.14 Eager and Graph Mode).

Currently the eager backend is build upon tf.keras.layer. Since TF 2.0 strengthens the usage of tf.keras, I would like to keep it that way, what do you think?

I don't know if we shoud really deprecate the graph mode completely, since it will still be faster than jit-compilation AFAIK.

justusschock avatar Jul 24 '19 09:07 justusschock

Reminder: Check if inheritance from tf.Module is better than inheritance from tf.keras.Layer [see #102 ]

justusschock avatar Aug 02 '19 11:08 justusschock

I think, we should wait with adressing this until #221 is merged, since this should make it easier to derive trainers

justusschock avatar Oct 06 '19 11:10 justusschock