Daniel Firu

icon company @quadric-io icon location Northern California

Results 1 issues of Daniel Firu

Add Support for Multi-Head Attention Operator

7 comment

# New Operator Self Attention ### Describe the operator Multi-headed attention is seeing prolific use in all transformers (mostly described in [pytorch](https://pytorch.org/docs/stable/generated/torch.nn.MultiheadAttention.html#torch.nn.MultiheadAttention)). Including native support for the op simplifies onnx...

operator