How to get hidden_states

[JCSTARS]

When I add output_hidden_states=True in generation config to get model hidden_states. It'snot working. After digging into modeling_internvl_chat.py, I found that the output_hidden_states seems cannot pass into self.language_model.generate. Is there some other ways?

generation_config = dict(max_new_tokens=4, do_sample=True, output_hidden_states=True) response = model.chat(tokenizer, pixel_values, question, generation_config)

[yuecao0119]

Hi,

Please try set "output_hidden_states": true, in config.json.

[Howuhh]

Hi @yuecao0119! I also need to get hidden states from v3 models. How should I modify the config? Can I do it without forking the official hf repo? Thanks!

P.S. I tried to supply output_hidden_states=True to the AutoConfig, but model.generate still returns just token indexes, not dict with hidden states.