[ENH] Implement AEAttentionBiGRUNetwork

[aadya940]

Implements Attention Bidirectional Gated Recurrent Unit based Auto Encoder .

[aeon-actions-bot[bot]]

Thank you for contributing to aeon

I have added the following labels to this PR based on the title: [ $\color{#FEF1BE}{\textsf{enhancement}}$ ]. I have added the following labels to this PR based on the changes made: [ $\color{#379E11}{\textsf{networks}}$ ]. Feel free to change these if they do not properly represent the PR.

The Checks tab will show the status of our automated tests. You can click on individual test runs in the tab or "Details" in the panel below to see more information if there is a failure.

If our pre-commit code quality check fails, any trivial fixes will automatically be pushed to your PR unless it is a draft.

Don't hesitate to ask questions on the aeon Slack channel if you have any.

[aadya940]

@hadifawaz1999 How can we use the Keras Subclassing API in this case, since the tests that don't have tensorflow will fail?

[MatthewMiddlehurst]

See the Hydra transformation for ideas, there is ways to get around it. Not sure you can do local classes in Python? Probably dont want to either way 🙂

[aadya940]

@MatthewMiddlehurst Thanks :-)

[aadya940]

@hadifawaz1999 What will change if we add support for temporal_latent_space in this? Do we apply the Attention and self._gate if temporal_latent_space is True since self._gate will Flatten the input tensor and convert it from 3 Dimensions to 2 Dimension and the output will no longer be temporal?

[hadifawaz1999]

See the Hydra transformation for ideas, there is ways to get around it. Not sure you can do local classes in Python? Probably dont want to either way 🙂

yup should work like we have in hydra

[hadifawaz1999]

@hadifawaz1999 What will change if we add support for temporal_latent_space in this? Do we apply the Attention and self._gate if temporal_latent_space is True since self._gate will Flatten the input tensor and convert it from 3 Dimensions to 2 Dimension and the output will no longer be temporal?

let me check the architecture and get back to you

[hadifawaz1999]

@hadifawaz1999 What will change if we add support for temporal_latent_space in this? Do we apply the Attention and self._gate if temporal_latent_space is True since self._gate will Flatten the input tensor and convert it from 3 Dimensions to 2 Dimension and the output will no longer be temporal?

let me check the architecture and get back to you

[hadifawaz1999]

sorry closed it by mistake miss clicking with my mouse :)

[hadifawaz1999]

if i understand correctly, the number of params for RNN layers depend on the batch size right ? as it depends on x.shape[0]

[aadya940]

Yep, that's what I understood.

[aadya940]

@hadifawaz1999 Can we really parametrize the number of layers here? Since, after the input passes through a Dense layers , its dimensions decrease from 3D to 2D and hence, we cant pass it to the next GRU layer. Also, the implementation here uses just 1 layer in the encoder, as well as the decoder: https://github.com/blafabregue/TimeSeriesDeepClustering/blob/main/networks/attention_rnn.py

[aadya940]

Could we just expand the dims of the tensor after each dense layer as follows?

if not (i == self.n_layers_encoder - 1):
   x = tf.expand_dims(x, -1)

[hadifawaz1999]

should merge main here too @aadya940

[aadya940]

#1962