Luca Soldaini

icon indicating a website link http://soldaini.net/ icon indicating an email [email protected]

icon company @allenai @queerai icon location Seattle, WA, USA icon location Research Scientist at @allenai on the OLMo team. LLM research, open-source AI.

Results 43 comments of Luca Soldaini

Generation of own dataset with Dolma Tokenizer CLI

If you run the python decoding code i shared above, what's your output? or do you get any error?

Adding script for processing many intermediate checkpoints at once for offline evals

@jenahwang would it be possible to merge in your changes to the other script? or consolidate the two? it's confusing to have two conversion scripts, and it doubles maintenance.

[Dolma Tutorial (https://allenai.github.io/docs): 707NotFound]

hi @yushengsu-thu , does the [Getting Started](https://github.com/allenai/dolma/blob/main/docs/getting-started.md) guide cover your needs? if not, let us know what's missing.