nnsight icon indicating copy to clipboard operation
nnsight copied to clipboard

Better error messages for accessing outside of trace context

Open MichaelRipa opened this issue 9 months ago • 4 comments

Description

This PR introduces more informative error messages when trying to access .input, .inputs or .output outside of a trace context.

MichaelRipa avatar Mar 24 '25 18:03 MichaelRipa

Did you run into limitations when you tried to decorate the functions in question? Is it because they're already decorated as properties?

AdamBelfki3 avatar Mar 24 '25 18:03 AdamBelfki3

@MichaelRipa Can you also make this error message better in this PR? https://github.com/ndif-team/nnsight/blob/149e14c12dde9267868051878e7091011b27fe62/src/nnsight/tracing/graph/node.py#L143

People have suggesting adding Did you forget to call .save()? or something

JadenFiotto-Kaufman avatar Mar 25 '25 11:03 JadenFiotto-Kaufman

@MichaelRipa Can you also make this error message better in this PR?

https://github.com/ndif-team/nnsight/blob/149e14c12dde9267868051878e7091011b27fe62/src/nnsight/tracing/graph/node.py#L143

People have suggesting adding Did you forget to call .save()? or something

Done. I feel like there might be somewhere better upstream to catch and log this (since .save() isn't well defined with respect to the Node class) but just directly updated the error message for the time being.

MichaelRipa avatar Mar 25 '25 13:03 MichaelRipa

@JadenFiotto-Kaufman The node error message shows up in undesirable circumstances:

import nnsight
model = nnsight.LanguageModel("openai-community/gpt2")

with model.trace("ayy"):
  x = model.transformer.h[0].attn.attn_dropout.output.save()
    
print(x.shape)
ValueError: Accessing value before it's been set. Did you forget to call .save()?

Is this expected behavior (and if so would there be a better error to log) or a bug?

MichaelRipa avatar Mar 27 '25 14:03 MichaelRipa