codebook-features

codebook-features copied to clipboard

Currently, there are two ways to format keys in codebook-related dictionaries: 1) "layer{x}_{cb_at}_gcb{y}" (adv name), and 2) "layer{x}_head{y}" (base name). The Second was introduced for convenience but shouldn't be used as it doesn't incorporate the cb_at parameter. So for a model, if in a given layer there are two places codebooks are applied at, the second way wouldn't be able to distinguish between the two codebooks. So this format can be removed and we can use 1) universally.