Clément Dumas

### Applies To - [x] Notebooks (.ipynb files) - [ ] Interactive Window and\/or Cell Scripts (.py files with \#%% markers) ### What happened? When I try to load this...

Currently i have to dispatch my language model before calling `model.load_adapter(adapter_id)`, otherwise i get ```py File "/mnt/nw/home/c.dumas/projects/diffing-toolkit/.venv/lib/python3.12/site-packages/safetensors/torch.py", line 381, in load_file with safe_open(filename, framework="pt", device=device) as f: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ safetensors_rust.SafetensorError: device...

In the dispatch function: https://github.com/ndif-team/nnsight/blob/40154765770f1c1697e79668a1454ba848dfe5ca/src/nnsight/modeling/mixins/meta.py#L58 We don't check if the model is already dispatched. Is this on purpose? Are there any reason not to check if `model.dispatched` is true and...

Unused renaming scheme make cache access via renamed name fail

9

Even if a renaming is not used (`model.language_model` here): ```py from nnsight import LanguageModel m = LanguageModel( "yujiepan/llama-3.2-tiny-random", rename={"model": "foo", "model.language_model": "foo"}, # rename={"model": "foo"}, # works ) with m.trace("Hello,...

Using `trace=False` is not working anymore. Idk if it's deprecated but not removed yet or if it's a bug. Tested on 0.5.7 and 0.5.10. ```py import torch as th import...

[0.5] nnsight actually depends on transformers >= 4.28 right now

1

`LlamaConfig` was only added to 4.28. It's probably fine to just change the dependancy, another option is to have a fallback if LlamaConfig doesn't exists. ```py nnterp/nnsight_utils.py:3: in from nnsight...

Would be cool if we could unpack saved shapes into save ints ```py from nnsight import LanguageModel model = LanguageModel("gpt2") with model.trace("hello") as tracer: batch_size, seq_len = model.inputs[1]["input_ids"].shape.save() # batch_size...

Cache fails to be displayed in notebook if using the path syntax and printing it prints the full cache

1

```py from nnsight import LanguageModel gpt2 = LanguageModel("gpt2", rename={".transformer.h": "layers"}) with gpt2.trace("Hello") as tracer: cache = tracer.cache(modules=[gpt2.layers[0]]).save() print(cache.keys()) print(f"type: {type(cache['model.transformer.h.0'])}") print(f"type: {type(cache.model.transformer.h[0])}") display(f"value: {cache['model.transformer.h.0']}") display(f"value: {cache.model.transformer.h[0]}") display(cache["model.transformer.h.0"]) display(cache.model.transformer.h[0]) # fails...

Would be nice to be able to run `nnsight login` and then be prompted to pas API key. Would be even nicer if we could do `from nnsight import login`...