Gary Linscott
Gary Linscott
Even more incredible, this allows me to run the full 65B model on a machine with 32GB of RAM quite quickly! ``` 7B - 2.93 seconds per pass - ETA...
> > Even more incredible, this allows me to run the full 65B model on a machine with 32GB of RAM quite quickly! > > wait... how does that work....
Very nice! On, 30B `8.58 seconds per pass` now, `10.99 seconds per pass` before.
I vote for this to be higher priority :). Bazel is commonly filling up hard drives on our project from the `~/.cache/bazel` directory growing unbounded.