text-generation-webui Feature Request: Generate and replicate seeds to reproduce text

Attempted to find the best method for generating and replicating seeds through 'server.py' and 'api-example.py' but cannot replicate text results or find where the scripts generate seeds.

Best example for generating and replicating seeds as per https://github.com/AUTOMATIC1111

Feature should appear on interface and same option using current 'api-example.py' and/or 'server.py' (unless there's a better way?)

Reason for request is to have the same level of output reproducability as per stable-diffusion-webui

The following are examples of many possible solutions.

Currently these snippets work in other text generation scripts but cannot find where to apply them in text-generation-webui.

generate best seeds here:

#torch.seed() #SEED=torch.seed() #print(SEED)

copy seed above here and comment out code above

seed = 6404812195821871557 print(seed) torch.manual_seed(seed) if torch.cuda.is_available(): torch.cuda.manual_seed_all(seed)

Cheers,

Mar 20 '23 21:03 GenTxt

Yeah same. I would love to get the possibility to manipulate the seed for tests, especially for the Lora VS non Lora one. https://github.com/oobabooga/text-generation-webui/issues/332#issuecomment-1475296375

Mar 21 '23 09:03 BadisG

Based on https://github.com/oobabooga/text-generation-webui/issues/332#issuecomment-1478008867 and https://github.com/oobabooga/text-generation-webui/issues/332#issuecomment-1478064078

We know how to add the seed on the repository, someone should make a PR to implement that onto the front-end I think.

Go to .\text-generation-webui\modules
Open up text_generation.py
Put this code just after "clear_torch_cache()." (Line 202)

seed = 12
torch.manual_seed(seed)
torch.cuda.manual_seed_all(seed)
print("seed: " + str(seed))

Mar 21 '23 16:03 BadisG

I tested this and it looked like it worked. I asked the same exact questions 3 timed and got the exact same answer three times. However it 'broke' chat logging and --verbose output, so I don't have verification.

Enabling --chat caused these errors:

To create a public link, set `share=True` in `launch()`.
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "C:\code\textgen_working\installer_files\env\lib\site-packages\uvicorn\protocols\websockets\websockets_impl.py", line 254, in run_asgi
    result = await self.app(self.scope, self.asgi_receive, self.asgi_send)
  File "C:\code\textgen_working\installer_files\env\lib\site-packages\uvicorn\middleware\proxy_headers.py", line 78, in __call__
    return await self.app(scope, receive, send)
  File "C:\code\textgen_working\installer_files\env\lib\site-packages\fastapi\applications.py", line 273, in __call__
    await super().__call__(scope, receive, send)
  File "C:\code\textgen_working\installer_files\env\lib\site-packages\starlette\applications.py", line 122, in __call__
    await self.middleware_stack(scope, receive, send)
  File "C:\code\textgen_working\installer_files\env\lib\site-packages\starlette\middleware\errors.py", line 149, in __call__
    await self.app(scope, receive, send)
  File "C:\code\textgen_working\installer_files\env\lib\site-packages\starlette\middleware\cors.py", line 76, in __call__
    await self.app(scope, receive, send)
  File "C:\code\textgen_working\installer_files\env\lib\site-packages\starlette\middleware\exceptions.py", line 79, in __call__
    raise exc
  File "C:\code\textgen_working\installer_files\env\lib\site-packages\starlette\middleware\exceptions.py", line 68, in __call__
    await self.app(scope, receive, sender)
  File "C:\code\textgen_working\installer_files\env\lib\site-packages\fastapi\middleware\asyncexitstack.py", line 21, in __call__
    raise e
  File "C:\code\textgen_working\installer_files\env\lib\site-packages\fastapi\middleware\asyncexitstack.py", line 18, in __call__
    await self.app(scope, receive, send)
  File "C:\code\textgen_working\installer_files\env\lib\site-packages\starlette\routing.py", line 718, in __call__
    await route.handle(scope, receive, send)
  File "C:\code\textgen_working\installer_files\env\lib\site-packages\starlette\routing.py", line 341, in handle
    await self.app(scope, receive, send)
  File "C:\code\textgen_working\installer_files\env\lib\site-packages\starlette\routing.py", line 82, in app
    await func(session)
  File "C:\code\textgen_working\installer_files\env\lib\site-packages\fastapi\routing.py", line 289, in app
    await dependant.call(**values)
  File "C:\code\textgen_working\installer_files\env\lib\site-packages\gradio\routes.py", line 483, in join_queue
    if blocks.dependencies[event.fn_index].get("every", 0):
IndexError: list index out of range
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "C:\code\textgen_working\installer_files\env\lib\site-packages\uvicorn\protocols\websockets\websockets_impl.py", line 254, in run_asgi
    result = await self.app(self.scope, self.asgi_receive, self.asgi_send)
  File "C:\code\textgen_working\installer_files\env\lib\site-packages\uvicorn\middleware\proxy_headers.py", line 78, in __call__
    return await self.app(scope, receive, send)
  File "C:\code\textgen_working\installer_files\env\lib\site-packages\fastapi\applications.py", line 273, in __call__
    await super().__call__(scope, receive, send)
  File "C:\code\textgen_working\installer_files\env\lib\site-packages\starlette\applications.py", line 122, in __call__
    await self.middleware_stack(scope, receive, send)
  File "C:\code\textgen_working\installer_files\env\lib\site-packages\starlette\middleware\errors.py", line 149, in __call__
    await self.app(scope, receive, send)
  File "C:\code\textgen_working\installer_files\env\lib\site-packages\starlette\middleware\cors.py", line 76, in __call__
    await self.app(scope, receive, send)
  File "C:\code\textgen_working\installer_files\env\lib\site-packages\starlette\middleware\exceptions.py", line 79, in __call__
    raise exc
  File "C:\code\textgen_working\installer_files\env\lib\site-packages\starlette\middleware\exceptions.py", line 68, in __call__
  File "C:\code\textgen_working\installer_files\env\lib\site-packages\fastapi\middleware\asyncexitstack.py", line 21, in __call__
    raise e
  File "C:\code\textgen_working\installer_files\env\lib\site-packages\fastapi\middleware\asyncexitstack.py", line 18, in __call__
    await self.app(scope, receive, send)
  File "C:\code\textgen_working\installer_files\env\lib\site-packages\starlette\routing.py", line 718, in __call__
    await route.handle(scope, receive, send)
  File "C:\code\textgen_working\installer_files\env\lib\site-packages\starlette\routing.py", line 341, in handle
    await self.app(scope, receive, send)
  File "C:\code\textgen_working\installer_files\env\lib\site-packages\starlette\routing.py", line 82, in app
    await func(session)
  File "C:\code\textgen_working\installer_files\env\lib\site-packages\fastapi\routing.py", line 289, in app
    await dependant.call(**values)
  File "C:\code\textgen_working\installer_files\env\lib\site-packages\gradio\routes.py", line 483, in join_queue
    if blocks.dependencies[event.fn_index].get("every", 0):
IndexError: list index out of range

Mar 21 '23 17:03 ill13

New PR to implement manual seed #495

Mar 22 '23 15:03 wywywywy

PR is now merged

Mar 22 '23 18:03 wywywywy

PR is now merged

Great job! Seed is an important parameter and I'm happy I can manipulate it now 😄

Mar 22 '23 18:03 BadisG

Dumb question. How can I find the seed that was used for the generation of text that I like?

Mar 23 '23 22:03 ndkling

Thanks for the PR and merge. Same seeds = same output (for the most part but there are quirky exceptions - another issue)

The only missing piece is how to print the random seed being generated from 'api-example.py'

'seed': -1,

That would answer ndkling's question and mine. We can close the issue then.

Cheers

Mar 24 '23 15:03 GenTxt

Fixed here:

https://github.com/mcmonkey4eva/text-generation-webui/tree/seed-fix

Thanks to both

Apr 12 '23 18:04 GenTxt