banyan-god

Is this loss curve normal

21

 I am running on 2x 4090 , updated gpu to 2 instead of 8 in gradient_accumulation_steps ``` more train_gpt2.py # config for...

This Dockerfile sets up environment to start the training process. You still need to download and run `python dev/data/fineweb.py --version 10B` You can build the dockerfile using `docker build -t...

RoPE has precision errors when used with BFloat16

1

The current implementation converts trigonometric values (cos_cached and sin_cached) to bfloat16, which introduces significant precision issues. This degrades the relative positional encoding properties of RoPE, particularly in **long-context** scenarios, as...

Problem Currently, the Pyctuator constructor requires a registration_url parameter and always attempts to register with Spring Boot Admin via this URL. However, when using [Kubernetes discovery](https://codecentric.github.io/spring-boot-admin/current/#kubernetes) in Spring Boot Admin,...