Clément Dumas

Error when using stopping_criteria in .generate if remote=True

5

Remote execution does not support stopping_criteria right now: ```py from nnsight import LanguageModel from transformers import StoppingCriteria class Stopping(StoppingCriteria): def __init__(self): pass def __call__(self, input_ids, _scores, **_kwargs): return False #...

[Feature Request] Add Stopping Criteria support

2

Hey, Huggingface offers a useful [Stopping Criteria class](https://huggingface.co/docs/transformers/internal/generation_utils#transformers.StoppingCriteria) that can be utilized in the `.generate` function to halt generation based on custom conditions. Do you think it would be possible...

Space after unnormalized token is added when `use_fast=True` for Llama tokenizer

10

Related to: https://github.com/huggingface/transformers/issues/25073 In my current project, I'd like to add a special token that doesn't insert a space to the next token. Currently, I need to specify `use_fast=False` in...

Add a rename_modules_dict arg to Envoy that allows renaming modules

3

After discussing with @cadentj some alternatives to transformer_lens (which sometimes has a significant implementation gap with HuggingFace), he suggested adding a renaming feature to nnsight itself, as it might be...

That would be nice to allow us to use bigger batch size when we don't need the gradient

Sometimes when I go to https://nnsight.net/status/, I can't see available models. Looking at the web console I'm getting those erros: ``` (3)(+0001387): Error: HTTP request to http://127.0.0.1:23119/connector/ping rejected with status...

```py from transformers import AutoModelForCausalLM, AutoTokenizer from nnsight import LanguageModel model_name = "Maykeye/TinyLLama-v0" tokenizer = AutoTokenizer.from_pretrained(model_name) model = AutoModelForCausalLM.from_pretrained(model_name) nn_model_from_hf = LanguageModel(model, tokenizer=tokenizer) nn_model = LanguageModel(model_name) print(type(nn_model.config), type(nn_model_from_hf.config), type(nn_model_from_hf._model.config)) ```...

Inconsistent behavior with module renaming in NNsight

6

When using the new `rename` feature in NNsight, I've discovered some inconsistent behaviors while working on unit tests: ```python from nnsight import LanguageModel import torch as th gpt2 = LanguageModel("gpt2",...

Consider the following tested on 0.4.5 and `dev` branch: ```py from nnsight import LanguageModel from torch.utils.data import DataLoader import nnsight as nns model = LanguageModel("Maykeye/TinyLLama-v0", device_map="auto") prompts = ["Hello, world!",...

Disclaimer: I was not surprised that this fails as it doesn't make sense to use all and next on the same module ```py from nnsight import LanguageModel model = LanguageModel("gpt2")...