LWM

LWM copied to clipboard

Request for guidance on fine-tuning a model with custom data

2

Hello, I am a big fan of your project and I am interested in using your model for my own data. However, I am new to fine-tuning models and I...

awzhgw

Great work! Any plan to train a smaller version, e.g. around 3B?

4

Hello, It's a really great work which contributes a lot to the community! Do you have any plan to train a smaller version of large world model (e.g., 1~3B), which...

StarCycle

Why always use float32 precision in training?

1

Is it necessary to use float32 in training? Why not use the widely used bf16 type, which saves more gpu memory. Looking forward to your reply, Thansk!

cizhenshi

Any experiments on reasoning or world modeling ?

1

This paper talks about long context length for a language model which is extended to be a vision-language model. I wonder why is it called World Model. It is not...

Randolph-zeng

awzhgw

awzhgw

cyj95

Pytorch version of the vision model

4

If possible please provide a pytorch version of the vision model.

aamir-gmail

out of memory error

3

`bash scripts/run_vision_chat.sh` removed --mesh_dim param model is [LWM-Chat-32K-Jax](https://huggingface.co/LargeWorldModel/LWM-Chat-32K-Jax) out of memory error, how to solve it my card is nvidia 2080 super 8G ``` WARNING: All log messages before absl::InitializeLog()...

lucky2046

bash run_vision_chat.sh -- cause flax.errors.ScopeParamNotFoundError: Could not find parameter named "embedding" in scope "/transformer/wte"

7

While run the command of "bash scripts/run_vision_chat.sh". Error happended .How to fix it. ===================================================================== (lwm) llm@llm-PowerEdge-R730xd:~/projects/LWM-main$ bash scripts/run_vision_chat.sh I0221 14:02:43.257625 139932541391232 xla_bridge.py:660] Unable to initialize backend 'rocm': NOT_FOUND: Could not...

jackyin68