bumblebee icon indicating copy to clipboard operation
bumblebee copied to clipboard

Published 20 hours ago • verified publisher icon elixir-nx

Add annotations to QKV layers

Open seanmor5 opened this issue 2 years ago • 0 comments
trafficstars

Related to https://github.com/elixir-nx/axon/issues/549

LoRA targets QKV nodes. Right now the only way to do it is by inferring based on layer name. We can provide more information by giving specific metadata (such as op_name) for which layers are QKV

seanmor5 avatar Nov 20 '23 22:11 seanmor5