Failed to build the chat prompt.

[thejohnd0e]

Describe the bug

Failed to build the chat prompt.

Is there an existing issue for this?

• [X] I have searched the existing issues

Reproduction

Failed to build the chat prompt.

Screenshot

Logs

16:40:56-190590 ERROR    Failed to build the chat prompt. The input is too long for the
                         available context length.

                         Truncation length: 4096
                         max_new_tokens: 4096 (is it too high?)
                         Available context length: 0

Traceback (most recent call last):
  File "R:\text-generation-webui-main\installer_files\env\Lib\site-packages\gradio\queueing.py", line 566, in process_events
    response = await route_utils.call_process_api(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "R:\text-generation-webui-main\installer_files\env\Lib\site-packages\gradio\route_utils.py", line 261, in call_process_api
    output = await app.get_blocks().process_api(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "R:\text-generation-webui-main\installer_files\env\Lib\site-packages\gradio\blocks.py", line 1786, in process_api
    result = await self.call_function(
             ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "R:\text-generation-webui-main\installer_files\env\Lib\site-packages\gradio\blocks.py", line 1350, in call_function
    prediction = await utils.async_iteration(iterator)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "R:\text-generation-webui-main\installer_files\env\Lib\site-packages\gradio\utils.py", line 583, in async_iteration
    return await iterator.__anext__()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "R:\text-generation-webui-main\installer_files\env\Lib\site-packages\gradio\utils.py", line 576, in __anext__
    return await anyio.to_thread.run_sync(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "R:\text-generation-webui-main\installer_files\env\Lib\site-packages\anyio\to_thread.py", line 56, in run_sync
    return await get_async_backend().run_sync_in_worker_thread(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "R:\text-generation-webui-main\installer_files\env\Lib\site-packages\anyio\_backends\_asyncio.py", line 2144, in run_sync_in_worker_thread
    return await future
           ^^^^^^^^^^^^
  File "R:\text-generation-webui-main\installer_files\env\Lib\site-packages\anyio\_backends\_asyncio.py", line 851, in run
    result = context.run(func, *args)
             ^^^^^^^^^^^^^^^^^^^^^^^^
  File "R:\text-generation-webui-main\installer_files\env\Lib\site-packages\gradio\utils.py", line 559, in run_sync_iterator_async
    return next(iterator)
           ^^^^^^^^^^^^^^
  File "R:\text-generation-webui-main\installer_files\env\Lib\site-packages\gradio\utils.py", line 742, in gen_wrapper
    response = next(iterator)
               ^^^^^^^^^^^^^^
  File "R:\text-generation-webui-main\modules\chat.py", line 414, in generate_chat_reply_wrapper
    for i, history in enumerate(generate_chat_reply(text, state, regenerate, _continue, loading_message=True, for_ui=True)):
  File "R:\text-generation-webui-main\modules\chat.py", line 382, in generate_chat_reply
    for history in chatbot_wrapper(text, state, regenerate=regenerate, _continue=_continue, loading_message=loading_message, for_ui=for_ui):
  File "R:\text-generation-webui-main\modules\chat.py", line 319, in chatbot_wrapper
    prompt = apply_extensions('custom_generate_chat_prompt', text, state, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "R:\text-generation-webui-main\modules\extensions.py", line 231, in apply_extensions
    return EXTENSION_MAP[typ](*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "R:\text-generation-webui-main\modules\extensions.py", line 107, in _apply_custom_generate_chat_prompt
    return extension.custom_generate_chat_prompt(text, state, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "R:\text-generation-webui-main\extensions\example\script.py", line 109, in custom_generate_chat_prompt
    result = chat.generate_chat_prompt(user_input, state, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "R:\text-generation-webui-main\modules\chat.py", line 223, in generate_chat_prompt
    raise ValueError
ValueError

System Info

Processor	11th Gen Intel(R) Core(TM) i7-11700 @ 2.50GHz   2.50 GHz
Installed RAM	128 GB (128 GB usable)
System type	64-bit operating system, x64-based processor
RTX 3060

[D-Vaillant]

This is happening to me too on Windows with a 4090 when I try to use the OpenAI API. The WebUI itself is working fine. Tried LLama3-8B with exLlama and GGUF and neither worked properly.

Similar issue on WSL.

[Enchante503]

Same here. Generate in NoteBook and Default works fine.

ERROR    Failed to build the chat prompt. The input is too long for the available context length.

                         Truncation length: 0
                         max_new_tokens: 512 (is it too high?)
                         Available context length: -512

[AbstractEyes]

Have the same problem with some linux based A40s and it's well into September.