What's the equivalence of tf.contrib.legacy_seq2seq.attention_decoder in tensorflow2?

[JiashuGuo]

Describe the feature and the current behavior/state.

I have to get a model that was built with tf1 to run with tf2 but didn't find the equivalence of the function tf.contrib.legacy_seq2seq.attention_decoder in the tf_addons seq2seq API, for example, how do you upgrade this functions in tf2? Is there any function that does the same work as attention_decoder or do I need to write an attention decoder myself?

tf.contrib.legacy_seq2seq.attention_decoder(
        decoder_inputs=decoder_inputs,
        initial_state=initial_state,
        attention_states=self._net,
        cell=cell,
        loop_function=self.get_input)

Which API type would this fall under (layer, metric, optimizer, etc.) tfa.seq2seq

[guillaumekln]

There is no direct replacement for this function but there are modules that achieve the same thing.

• You can extend your RNN cell with an attention mechanism using tfa.seq2seq.AttentionWrapper
• You can create a decoder with the RNN cell using tfa.seq2seq.BasicDecoder

There are small examples in these pages which should get you started with these modules.

[JiashuGuo]

Thanks for the reply, do you know any other more detailed examples or docs? I didn't find the Returns in the document.

[guillaumekln]

You can also find an example in the module README which compares implementations with TensorFlow 1 and TensorFlow 2: https://github.com/tensorflow/addons/tree/master/tensorflow_addons/seq2seq#sample-code-and-migration-guide-from-tf-1x

[skwolvie]

hi @JiashuGuo, I'm dabbling with the same procedure, and I'm unable to get ahead. Did you have any success? Please share!

[JiashuGuo]

hi @JiashuGuo, I'm dabbling with the same procedure, and I'm unable to get ahead. Did you have any success? Please share!

Hi, it has been quite a while since I worked on this project, as I remember I managed to make the code ran in tf2 based on some workaround I found online, it just didn’t report any error but the model actually didn’t has correct structures as it need in tf2, so the model failed to be trained in the next step. And after that I gave up on this project. So, I would say I hadn’t got any working solution on this problem…Good luck in your digging though!

[seanpmorgan]

TensorFlow Addons is transitioning to a minimal maintenance and release mode. New features will not be added to this repository. For more information, please see our public messaging on this decision: TensorFlow Addons Wind Down

Please consider sending feature requests / contributions to other repositories in the TF community with a similar charters to TFA: Keras Keras-CV Keras-NLP