Easy-Transformer
Easy-Transformer copied to clipboard
add better model properties table to docs
[WIP]
Description
This PR ports features from my transformerlens-model-table repo to TransformerLens, implementing many of the features requested in #97. I still need some feedback on this, and presumably building docs will fail for one reason or another once I make the PR.
Features:
The static table has a few more fields added to it, but the primary focus is the interactive table. This provides:
- information on parallel attn/mlps, positional embeddings, and other config elements
- filtering and searching on any column (i.e. sort by parameter count and only show standard positional embeddings)
- links back to the huggingface model page, where applicable (extracted from the "official model name")
- tokenizer information, including vocab hash (need feedback on if there is a better way to do this)
- full config in title text or new window
- organized view of dimensions of all tensors in state dict and activation cache (via setting device to
meta, doesn't require actually loading models)
Adds dependencies
under group docs:
Type of change
- [x] This change requires a documentation update
Screenshots
Before:
Original model properties table
After (static):
You can see what the generated data looks like here
After (interactive):
See demo
Checklist:
(currently draft PR, testing incomplete)
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [ ] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] I have not rewritten tests relating to key interfaces which would affect backward compatibility