codebook-features icon indicating copy to clipboard operation
codebook-features copied to clipboard

Remove base name and referencing codebooks using "layer{x}_head{y}" syntax

Open taufeeque9 opened this issue 1 year ago • 0 comments

Currently, there are two ways to format keys in codebook-related dictionaries: 1) "layer{x}_{cb_at}_gcb{y}" (adv name), and 2) "layer{x}_head{y}" (base name). The Second was introduced for convenience but shouldn't be used as it doesn't incorporate the cb_at parameter. So for a model, if in a given layer there are two places codebooks are applied at, the second way wouldn't be able to distinguish between the two codebooks. So this format can be removed and we can use 1) universally.

taufeeque9 avatar Sep 27 '23 20:09 taufeeque9