When training, LlamaCppModel.encode() got an unexpected keyword argument 'truncation'

[thelabcat]

Describe the bug

I cannot train a LLaMa LoRA because of what appears to be a coding mistake.

Is there an existing issue for this?

• [X] I have searched the existing issues

Reproduction

1. Load LLaMa 2 13b with the llama.cpp loader. I used 16 or 8 GPU layers, Tensorcores enabled. Other settings were default.
2. Create a raw text training file, save it to training/datasets, and load it in the Training tab.
3. Give the new LoRA a name.
4. Click "Start LoRA Training". After a few seconds, the WebUI will show (ERROR) and the included traceback is printed to the console.

Screenshot

No response

Logs

15:01:18-914753 WARNING  LoRA training has only currently been validated for    
                         LLaMA, OPT, GPT-J, and GPT-NeoX models. (Found model   
                         type: LlamaCppModel)                                   
15:01:23-916330 INFO     Loading raw text file dataset                          
Traceback (most recent call last):
  File "/home/wilbur/text-generation-webui/installer_files/env/lib/python3.11/site-packages/gradio/queueing.py", line 561, in process_events
    response = await route_utils.call_process_api(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/wilbur/text-generation-webui/installer_files/env/lib/python3.11/site-packages/gradio/route_utils.py", line 260, in call_process_api
    output = await app.get_blocks().process_api(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/wilbur/text-generation-webui/installer_files/env/lib/python3.11/site-packages/gradio/blocks.py", line 1741, in process_api
    result = await self.call_function(
             ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/wilbur/text-generation-webui/installer_files/env/lib/python3.11/site-packages/gradio/blocks.py", line 1308, in call_function
    prediction = await utils.async_iteration(iterator)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/wilbur/text-generation-webui/installer_files/env/lib/python3.11/site-packages/gradio/utils.py", line 575, in async_iteration
    return await iterator.__anext__()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/wilbur/text-generation-webui/installer_files/env/lib/python3.11/site-packages/gradio/utils.py", line 568, in __anext__
    return await anyio.to_thread.run_sync(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/wilbur/text-generation-webui/installer_files/env/lib/python3.11/site-packages/anyio/to_thread.py", line 56, in run_sync
    return await get_async_backend().run_sync_in_worker_thread(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/wilbur/text-generation-webui/installer_files/env/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 2134, in run_sync_in_worker_thread
    return await future
           ^^^^^^^^^^^^
  File "/home/wilbur/text-generation-webui/installer_files/env/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 851, in run
    result = context.run(func, *args)
             ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/wilbur/text-generation-webui/installer_files/env/lib/python3.11/site-packages/gradio/utils.py", line 551, in run_sync_iterator_async
    return next(iterator)
           ^^^^^^^^^^^^^^
  File "/home/wilbur/text-generation-webui/installer_files/env/lib/python3.11/site-packages/gradio/utils.py", line 734, in gen_wrapper
    response = next(iterator)
               ^^^^^^^^^^^^^^
  File "/home/wilbur/text-generation-webui/modules/training.py", line 456, in do_train
    train_data = Dataset.from_list([tokenize(x) for x in text_chunks])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/wilbur/text-generation-webui/modules/training.py", line 456, in <listcomp>
    train_data = Dataset.from_list([tokenize(x) for x in text_chunks])
                                    ^^^^^^^^^^^
  File "/home/wilbur/text-generation-webui/modules/training.py", line 374, in tokenize
    input_ids = encode(prompt, True)
                ^^^^^^^^^^^^^^^^^^^^
  File "/home/wilbur/text-generation-webui/modules/training.py", line 362, in encode
    result = shared.tokenizer.encode(text, truncation=True, max_length=cutoff_len)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: LlamaCppModel.encode() got an unexpected keyword argument 'truncation'

System Info

OS: Fedora 39 x86-64
GPU: NVidia RTX 2060 SUPER

[potesd]

I'm getting the same error.

[v1-valux]

Same error here.

model tab:

• Model leo-hessianai-7B-chat-bilingual-GGUF (llama2) with llama.cpp.
• 20 GPU layers, tested with Tensorcores enabled/disabled, Other settings default.

training-tab:

• Tested with Cutoff Length of 512 and/or fitted dataset to try to workaround truncation of data
• Used following custom data-formats. Other settings default.

Data Format according to model-page on huggingface

{
    "reply,prompt,system_message": "<|im_start|>system\n%system_message%<|im_end|>\n<|im_start|>user\n%prompt%<|im_end|>\n<|im_start|>assistant\n%reply%",
    "reply,prompt": "<|im_start|>user\n%prompt%<|im_end|>\n<|im_start|>assistant\n%reply%"
}

Dataset Format according to the format-file:

[{
"prompt": "some prompt",
"reply": "text"
}]

Logs:

Traceback (most recent call last):
  File "text-generation-webui/installer_files/env/lib/python3.11/site-packages/gradio/queueing.py", line 566, in process_events
    response = await route_utils.call_process_api(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "text-generation-webui/installer_files/env/lib/python3.11/site-packages/gradio/route_utils.py", line 261, in call_process_api
    output = await app.get_blocks().process_api(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "text-generation-webui/installer_files/env/lib/python3.11/site-packages/gradio/blocks.py", line 1786, in process_api
    result = await self.call_function(
             ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "text-generation-webui/installer_files/env/lib/python3.11/site-packages/gradio/blocks.py", line 1350, in call_function
    prediction = await utils.async_iteration(iterator)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "text-generation-webui/installer_files/env/lib/python3.11/site-packages/gradio/utils.py", line 583, in async_iteration
    return await iterator.__anext__()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "text-generation-webui/installer_files/env/lib/python3.11/site-packages/gradio/utils.py", line 576, in __anext__
    return await anyio.to_thread.run_sync(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "text-generation-webui/installer_files/env/lib/python3.11/site-packages/anyio/to_thread.py", line 56, in run_sync
    return await get_async_backend().run_sync_in_worker_thread(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "text-generation-webui/installer_files/env/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 2134, in run_sync_in_worker_thread
    return await future
           ^^^^^^^^^^^^
  File "text-generation-webui/installer_files/env/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 851, in run
    result = context.run(func, *args)
             ^^^^^^^^^^^^^^^^^^^^^^^^
  File "text-generation-webui/installer_files/env/lib/python3.11/site-packages/gradio/utils.py", line 559, in run_sync_iterator_async
    return next(iterator)
           ^^^^^^^^^^^^^^
  File "text-generation-webui/installer_files/env/lib/python3.11/site-packages/gradio/utils.py", line 742, in gen_wrapper
    response = next(iterator)
               ^^^^^^^^^^^^^^
  File "text-generation-webui/modules/training.py", line 483, in do_train
    train_data = data['train'].map(generate_and_tokenize_prompt, new_fingerprint='%030x' % random.randrange(16**30))
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "text-generation-webui/installer_files/env/lib/python3.11/site-packages/datasets/arrow_dataset.py", line 602, in wrapper
    out: Union["Dataset", "DatasetDict"] = func(self, *args, **kwargs)
                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "text-generation-webui/installer_files/env/lib/python3.11/site-packages/datasets/arrow_dataset.py", line 567, in wrapper
    out: Union["Dataset", "DatasetDict"] = func(self, *args, **kwargs)
                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "text-generation-webui/installer_files/env/lib/python3.11/site-packages/datasets/arrow_dataset.py", line 3161, in map
    for rank, done, content in Dataset._map_single(**dataset_kwargs):
  File "text-generation-webui/installer_files/env/lib/python3.11/site-packages/datasets/arrow_dataset.py", line 3522, in _map_single
    example = apply_function_on_filtered_inputs(example, i, offset=offset)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "text-generation-webui/installer_files/env/lib/python3.11/site-packages/datasets/arrow_dataset.py", line 3421, in apply_function_on_filtered_inputs
    processed_inputs = function(*fn_args, *additional_args, **fn_kwargs)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "text-generation-webui/modules/training.py", line 479, in generate_and_tokenize_prompt
    return tokenize(prompt, add_eos_token)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "text-generation-webui/modules/training.py", line 364, in tokenize
    input_ids = encode(prompt, True)
                ^^^^^^^^^^^^^^^^^^^^
  File "text-generation-webui/modules/training.py", line 352, in encode
    result = shared.tokenizer.encode(text, truncation=True, max_length=cutoff_len)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: LlamaCppModel.encode() got an unexpected keyword argument 'truncation'

System Info:

OS: Manjaro x86_64 Linux 6.6.32-1
GPU: NVIDIA RTX 3070