grok-1

grok-1 copied to clipboard

Optimize Error Handling and Regex Caching in Tensor Loading

5

# This PR introduces two key enhancements to the tensor loading process: (Fixes #220) - Improved error handling within ThreadPoolExecutor to provide detailed logs for failures during parallel tensor loading....

Madhav-MKNC

# Description The tensor loading process, specifically within ThreadPoolExecutor and regex operations in get_load_path_str, requires enhancements to improve error handling and efficiency. ## Enhanced Error Handling in ThreadPoolExecutor: Current behavior...

Madhav-MKNC

Nr. of devices needed

36

Running `python run.py` on a single Nvidia GPU it fails with `ValueError: Number of devices 1 must equal the product of mesh_shape (1, 8)` Can the nr of devices be...

zcobol

Should the version of jax in requirements.txt be cuda12-pip instead of cuda12_pip?

2

run `pip install -r requirements.txt` failed, replaced it as `cuda12-pip` successed.

zhangjh

Hyperbolic tangent attention squashing, attention kernels

2

The attention formula has an [unusual line](https://github.com/xai-org/grok-1/blob/d6d9447e2d3c9bd81da571dc8681ea60009e4c03/model.py#L865) that puts attention weights through `tanh`. 1. What is the reason for this? Normalization/entropy control? Is there a paper about it? 2. Would...

neverix

[type:refactor] add issue template

1

Yeah, I think grok-1 needs an issue template to avoid confused issue

moremind

Error when installing requirements

24

i have installed python 3.10 and venv. Trying to "pip install -r requirements.txt" ERROR: Ignored the following versions that require a different python version: 1.6.2 Requires-Python >=3.7,=3.7,=3.7,=3.7,

kesevone

Convert to pytorch model to use transformers from huggingface

10

Can someone covert this jax model to pytorch model implemented in transformers?

wenmengzhou

Huggingface download error

5

When I'm trying to download the weights from huggingface `huggingface-cli download xai-org/grok-1 --repo-type model --include ckpt-0/* --local-dir checkpoints --local-dir-use-symlinks False` I'm getting error `no matches found: ckpt-0/*`

mei-chen

Error "Could not find a version that satisfies the requirement jaxlib==0.4.25+cuda12.cudnn89"

20

Hello everyone. I've tried to run the pip install, but I'm facing the following error: ```bash ERROR: Could not find a version that satisfies the requirement jaxlib==0.4.25+cuda12.cudnn89; extra == "cuda12_pip"...

BIEMAX