ng-video-lecture

ng-video-lecture copied to clipboard

"index out of range" error when using a different embedding dimension than vocab_size

1

https://github.com/karpathy/ng-video-lecture/blob/52201428ed7b46804849dea0b3ccf0de9df1a5c3/bigram.py#L66 If I change the 2nd parameter (dimension of the embedding) to something different than `vocab_size` (e.g. 128), I got "index out of range error" in `generate()`. To replicate the...

zhoupingjay

Call `model.eval()` before generating?

1

I understand why we have to call `model.eval()` before calculating the average loss in `estimate_loss()`. But should we not similarly call `model.eval()` before we start generating from the model?

gustavdelius

updated dictionary names

1

Using ch is more consistent with the variable name the comprehension already uses internally, which is ch. Also adds more meaning and description, which is the purpose of a name....

caaker

time series data like BTC price

1

if we have a time series data like BTC price, then we don't need to do `token_embedding` i suppose, how do we do `position_embedding` in this case?

tesla-cat

patchy631

This is essentially a repeat of #15 but I'm making a new issue for visibility. It's noted that the license is MIT in the README, but there is no MIT...

matthewcarbone

In the code below, the query-key dot product is normalized by multiplying by the square root of the head size: https://github.com/karpathy/ng-video-lecture/blob/52201428ed7b46804849dea0b3ccf0de9df1a5c3/gpt.py#L83 Should we not be dividing instead? As seen in...

tylerkastner