Pete Walsh

Hey @borgr, at the moment the biggest obstacle is accessing the preprocessed training data. Without that you'd have to preprocess it on your own using tools in Dolma, which takes...

Hey @andrewfr could you please provide a code snippet to reproduce along with the output of `pip freeze`?

Hey @andrewfr, the 7B model requires about 27.6GB of GPU memory just to load it, and more to actually run inference. So you'd probably need a 40GB GPU. However you...

In your example just change "allenai/OLMo-7B" to "allenai/OLMo-1B"

@andrewfr it's okay to ignore those warnings. I'm not sure about that error. I'm guessing an update to `datasets` broke that loading script.

Hey @tanaymeh can you post the full traceback and the exact command you ran? Thanks

@tanaymeh, oh I think the issue is that the default value for `--tokenizer` is a relative path, relative to the root of the OLMo repo. So if you run from...

Fixed in 80db5e3d.

@dirkgr does it crash on `main`? I remember making a fix recently for loading the uninitialized optimizer state.

Yes, but the ablations are still going