LLMs-from-scratch

LLMs-from-scratch copied to clipboard

Bug description

pos_embedding_layer and token_embedding_layer definition in exercise_solutions chapter 2 incorrect

What operating system are you using?

Linux

Where do you run your code?

Local (laptop, desktop)

Environment

token_embedding_layer = torch.nn.Embedding(context_length, output_dim)
pos_embedding_layer = torch.nn.Embedding(vocab_size, output_dim)