layernorm topic

List layernorm repositories

LayerNorm_GRU

32
Stars
7
Forks
Watchers

Implement layer normalization GRU in pytorch

verified publisher icon ElektrischesSchaf

rmsnorm

207
Stars
11
Forks
Watchers

Root Mean Square Layer Normalization

verified publisher icon bzhangGo

layer_norm_expressivity_role

43
Stars
3
Forks
Watchers

Code for the paper "On the Expressivity Role of LayerNorm in Transformers' Attention" (Findings of ACL'2023)

verified publisher icon tech-srl

CUDA-Learn-Notes

1.2k
Stars
133
Forks
Watchers

🎉 Modern CUDA Learn Notes with PyTorch: fp32, fp16, bf16, fp8/int8, flash_attn, sgemm, sgemv, warp/block reduce, dot, elementwise, softmax, layernorm, rmsnorm.

verified publisher icon DefTruth