LLMs-from-scratch
LLMs-from-scratch copied to clipboard
Published
20 hours ago •
rasbt
rasbt
Fix: Typo in `appendix_d.py` comments.
In appendix_d.py the phrase "which lead to" should be "which leads to".
Applied fixes: --- a/pkg/llms_from_scratch/appendix_d.py +++ b/pkg/llms_from_scratch/appendix_d.py @@ -76,7 +76,7 @@ if global_step > warmup_steps: torch.nn.utils.clip_grad_norm_(model.parameters(), max_norm=1.0) else:
-
if global_step >= warmup_steps: # the book originally used global_step > warmup_steps, which lead to a skipped clipping step after warmup
-
if global_step >= warmup_steps: # the book originally used global_step > warmup_steps, which leads to a skipped clipping step after warmup torch.nn.utils.clip_grad_norm_(model.parameters(), max_norm=1.0) optimizer.step()
