[Docs] Clarify how Tokenizer.pad_to_multiple_of is useful in allowing use of GPU tensor cores

[ldorigo]

Hi! I'm not entirely sure of the best place to put this issue; the main Transformers repo links to https://github.com/huggingface/hub-docs/issues for issues about the website, but that seems to be for the Hub website, not the main huggingface documentation?

In the docs for Tokenizer.__call__() ( https://huggingface.co/docs/transformers/main/en/main_classes/tokenizer#transformers.PreTrainedTokenizer.call.pad_to_multiple_of ), you write:

pad_to_multiple_of (int, optional) — If set will pad the sequence to a multiple of the provided value. This is especially useful to enable the use of Tensor Cores on NVIDIA hardware with compute capability >= 7.5 (Volta).

I tried my best but couldn't find any additional information on how to do that. You could maybe consider adding a link to your own or external documentation on how to do that? (for reference, I read your article on optimizing for GPU and tried searching the docs for "Volta" but to no avail).

P.s.: unrelated, but the UX in your docs is absolutely stellar. Keep up the good work.

[sherlcok314159]

Maybe you can refer to this post.

[Narsil]

Very old issue that I somehow missed. sorry about that.

For future readers, nvidia drivers tend to prefer matrices of shapes of size multiple of eights including the sequence_length that this lib will produce.

However now in 2023, this is less and less true, newer drivers and cuda versions are smarter about this and will be able to use tensorcores even without this aligned padding. Your mileage may vary, and check is always better !

[github-actions[bot]]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.