OLMo icon indicating copy to clipboard operation
OLMo copied to clipboard
verified publisher icon allenai

Adding support for activation logging

Open ibeltagy opened this issue 2 years ago • 1 comments

🚀 The feature, motivation and pitch

No response

Alternatives

No response

Additional context

No response

ibeltagy avatar Sep 27 '23 00:09 ibeltagy

Draft PR for this is here: https://github.com/allenai/LLM/pull/330 Right now it's about 40% slower when you turn on activation logging (which is a config flag). I hope this can be optimized (e.g. by removing some .item() calls which load GPU tensors to CPU), but this is where it is right now.

saurabh111233212 avatar Oct 13 '23 16:10 saurabh111233212

Marking the items prior to Feb 29th as "closed".

dumitrac avatar Apr 30 '24 21:04 dumitrac