Missing LayerNormalization & MultiHeadAttention Keras Layers

[lqdev]

@Oceania2018 can you assign this issue to me. I'll try and take a stab at it.

I was trying to see whether implementing the Keras Transformer recommendation sample is feasible in .NET. However, I believe two of the layers, LayerNormalization and MultiHeadAttention are missing from the current implementation.

[GeorgeS2019]

@lqdev In your approach to MultiHeadAttention, perhaps you could share your feedback to this thread of discussion.

Perhaps we may find more people interested on this topics for .NET

[lindadamama]

missing layerNormalization However, Torchsharp's APIS are complete https://github.com/tensorflow/tensorflow/tree/v2.7.0/tensorflow/python/keras/layers/normalization https://github.com/dotnet/TorchSharp/tree/main/src/TorchSharp/NN/Normalization