Hard-coded llama `pad_token_id` in `examples/text-generation/utils.py`

[muhark]

System Info

- optimum-habana: 1.12.1

Information

• [X] The official example scripts
• [X] My own modified scripts

Tasks

• [X] An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• [X] My own task or dataset (give details below)

Reproduction

I noticed the issue when running the example code in examples/text-generation/text-generation-pipeline/README.md with llama-3.1-8b-instruct:

cd optimum-habana/examples/text-generation/text-generation-pipeline

python run_pipeline.py \
    --model_name_or_path meta-llama/Meta-Llama-3.1-8B-Instruct \
    --max_new_tokens 10 \
    --batch_size 4 \
    --max_input_tokens 128 \
    --bf16 \
    --use_hpu_graphs \
    --use_kv_cache

If you look at these lines:

https://github.com/huggingface/optimum-habana/blob/59d182dfa9a4c0ba068a1fd118b96b8ff0057c96/examples/text-generation/utils.py#L403C1-L417

You can see that pad_token_id, bos_token_id and eos_token_id are hardcoded to 0, 1 and 2.

In the vocab file for llama3.1, these correspond to the characters !, \ and #.

Expected behavior

I am not sure how we want to handle the lack of a pad token for llama in the official examples. I wanted to raise this to your attention in case it is something that you wanted to change going forward.