Question about thread safety

[miroljub1995]

We are thinking if we can reuse the single pipe, created on startup, between multiple threads from ThreadPoolExecutor. Is it thread safe?

[patrickvonplaten]

I think this really depends on how you use the pipeline. Note that schedulers which are part of the pipeline change the inner state quite a bit, therefore I don't think the pipeline is thread safe. You could however try to copy only the scheduler (it's very light weight and just a Python class) across threads.

Very curious to hear if "thread safety" is a common concern, issue -> we could think about a better API to ensure thread safety.

[patrickvonplaten]

For some more details this implementation should be thread safe:

from diffusers import DiffusionPipeline, StableDiffusionPipeline
from copy import deepcopy

pipe = DiffusionPipeline.from_pretrained("CompVis/stable-diffusion-v1-4")

components = pipe.components
components.pop("scheduler")
scheduler = pipe.scheduler

threaded_pipelines = []

num_threads = 10
for i in range(num_threads):
    scheduler = deepcopy(scheduler)
    threaded_pipelines.append(StableDiffusionPipeline(**components, scheduler=scheduler))

Here all the schedulers are copied and only the schedulers are stateful in inference, so this should work just fine at 0 memory overhead :-)

[nguyenmeteorops]

I have the same question, how to make sure pipeline generating thing in threadsafe ?

[patrickvonplaten]

Here we go: https://github.com/huggingface/diffusers/issues/1606#issuecomment-1346667389 ;-)

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[haoqiangyu]

Here we go: #1606 (comment) ;-)

I don't think this guarantees the thread safety of the pipeline

[haoqiangyu]

Here we go: #1606 (comment) ;-)

I don't think this guarantees the thread safety of the pipeline

If you use pytorch2.0.0 and above, it will cause the thread to freeze.