maxtext
maxtext copied to clipboard
google
A simple, performant and scalable Jax LLM!
This changes adds the following: - Allows creating on a monitor object that spins up a secondary "monitor & upload" thread to query Goodput of the job using the ml-goodput-measurement...
Added support for sliding window attention masking in TPU splash_attention
[DON'T MERGE] GCS Distributed Training Benchmark Infra + File-parallelism + Range-read Parquet files
# This is created as a draft PR for GCS internal members to comment. This will not be merged to main. ## File-parallelism + Range-read Parquet files This PR supports...
I have looked around for a script that could convert MaxText Gemma and Gemma 2 checkpoints to Hugging Face format but i have not find anything related. This may related...
- adding mixtral-8x22b config (to pyconfig as well) - improving the llama and mistral conversion script - in-place weight writing to reduce total RAM usage - better progress tracking
I am trying to adapt Llama3 for long context, is 128k. I am training on a v5-256, and are trying to follow the procedure explained in https://arxiv.org/pdf/2407.14482. Basically this states:...
The network release of 6/27 can be found in this doc: https://docs.google.com/document/d/1D5umT4-WDuNnYf3ieQ5SfLdmvGRPBQGLB662udzwz8I/edit?tab=t.0#bookmark=id.tezgud50glwu.
