Shao Tang

How to use it?

3

Not seen button in edit bar or any hotkey? After selecting the whole document, how exactly this plugin can be ran?

Add the hyperlink of the transfomer doc

1

Fixes #120488 - The shape for forward pass is clearly stated in the main [transformer class](https://pytorch.org/docs/stable/generated/torch.nn.Transformer.html) - Boolean mask for _key_padding_mask is also explained in the main transformer class. Therefore,...

error in running chapter-05.ipynb and chapter-06.ipynb

The doc build deployment has been failing since jan

1

## Context  ## Failure Logs [if any] fatal: unable to access 'https://github.com/pytorch/examples.git/': The requested URL returned error: 403 Running post deployment cleanup jobs… 🗑️ /usr/bin/git worktree remove github-pages-deploy-action-temp-deployment-folder --force...

1. fix the consistency of the transpose notation, use .T (pytorch convention) across the board and eliminates the mix of ^T 2. fix a space format

In the kernal ``` __global__ void add_bias(float* out, float* bias, int B, int T, int OC) { int idx = blockIdx.x * blockDim.x + threadIdx.x; int stride = blockDim.x *...

fix typo in crossentropy_foward.cu

Include the online softmax CPU code and a fully parallelized GPU kernal

4

1. Include the online softmax CPU code (from the paper [Online normalizer calculation for softmax](https://arxiv.org/pdf/1805.02867.pdf)). 2. Its native port to GPU kernel `kernel 5` (for education comparison). 3. Include the...

Fix the potential int overflow related to B*T*V

8

With ``` int B = 8; int T = 1024; int V = 50257; ``` The ratio of (8 * 1024 * 52507) to INT_MAX is: 0.200298. Therefore, if we...

https://github.com/karpathy/llm.c/issues/147 Code changes are tested by execution Scanned through the following files and touched 3 files (2 files are OK wrt constness) 1. classifier_fused.cu 2. crossentropy_foward.cu 3. crossentropy_softmax_backward.cu 4. gelu_forward...