UnifiedTransformer -> nnsight0.5
Reintroduce the implementation of the UnifiedTransformer class to work with the latest version nnsight. Add unit tests to validate this functionality.
Any updates here? We're using UnifiedTransformer in one of our baseline implementations for MIB, and it would be super helpful to have in a more recent version!
There's a discussion Jaden wants to have about whether an 0.5 feature (operation hooking) makes it unneeded to have UnifiedTransformer in this version.
Yeah I think it might makes more sense to implement a new class using renaming e.g. like this and predefine hook functions per model to grab attention etc rather than using TL as a backend.
Pro:
- Official impl -> no precision issue
Cons:
- If HF change the name of intermediate variable this would break while TL impl would remain more stable