Installation Steps

[Spr-Peach]

Sorting out the installation steps:

precondition:

download and install node.js from https://nodejs.org/en add "%NODE_PATH%","C:\Program Files\nodejs" & "C:\Program Files\nodejs\node_global" to system environment variable (by default)

then install pnpm:

npm install -g pnpm

installation:

clone to path: C:\Users\<your_username>:

cd C:\Users\<your_username>
git clone https://github.com/cumulo-autumn/StreamDiffusion.git
cd streamdiffusion

create python3.10 virtual environmen and activate it:

python -m venv streamdiffusion
streamdiffusion\scripts\activate.bat

install torch for cuda 11.8:

pip3 install torch==2.1.0 torchvision==0.16.0 xformers --index-url https://download.pytorch.org/whl/cu118

Install StreamDiffusion (latest version, recommended):

pip install git+https://github.com/cumulo-autumn/StreamDiffusion.git@main#egg=streamdiffusion[tensorrt]

Install TensorRT extension:

python -m streamdiffusion.tools.install-tensorrt

examples:

screen function: installation: navigate to examples folder and install requirements:

cd examples
pip install -r screen/requirements.txt

measure the performance of streamdiffusion

python benchmark/multi.py

If everything goes well press Ctrl+C to exit the current running. run:

python screen/main.py

If everything goes well press Ctrl+C to exit the current running.

img2img function (place source images in path" ./images/input/", current path: C:\Users<user name>\StreamDiffusion\examples): Image-to-image for a single image:

python img2img/single.py --input ../images/inputs/input.png --output ../images/outputs/output.png

If everything goes well press Ctrl+C to exit the current running. Image-to-image for multiple images:

python img2img/multi.py --input ../images/inputs --output-dir ../images/outputs

If everything goes well press Ctrl+C to exit the current running.

txt2img (current path: C:\Users\\StreamDiffusion\examples): text-to-image for a single image:

python txt2img/single.py --output ../images/outputs/output.png --prompt "<your prompts>"

If everything goes well press Ctrl+C to exit the current running. text-to-image for multiple images:

python txt2img/multi.py --output ../images/outputs --prompt "<your prompts>"

If everything goes well press Ctrl+C to exit the current running.

vid2vid: install extra dependencies (current path: C:\Users\\StreamDiffusion\examples):

pip install -r vid2vid/requirements.txt

run (place source video in path" ./images/input/"):

python vid2vid/main.py --input ..images/inputs/input.mp4 --output ../images/outputs/output.mp4

If everything goes well press Ctrl+C to exit the current running.

other command line options:

1. --model_id_or_path: --model_id_or_path: "<the model ID in huggingface>" #hugging face --model_id_or_path: "<model path/model name>" #local, recommend
2. --lora_dict: --lora_dict "{'<lora_1 path>' : <lora_1 scale> , '<lora_2 path>' : <lora_2 scale> }"
3. --prompt: --prompt ""
4. --negative prompts: --negative_prompts ""

demo:

realtime-img2img: every single time to execute "realtime-img2img" function you have to run every line below from "npm i"

cd C:\Users\<user name>\StreamDiffusion\demo\realtime-img2img\frontend
npm i
npm run build
cd..
pip install -r requirements.txt

run (need long time for launch till webpage pop up automatically at "http://0.0.0.0:7860/"; there will be various errors messages just disregard it. see issues #130 for details):

python main.py  --acceleration tensorrt

If everything goes well press Ctrl+C to exit the current running.

realtime-txt2img:

cd C:\Users\<user name>\StreamDiffusion\demo\realtime-txt2img
pip install -r requirements.txt
cd frontend
pnpm i
pnpm run build
cd..

run:

python main.py

If everything goes well press Ctrl+C to exit the current running.

vid2vid:

cd C:\Users\<user name>\StreamDiffusion\demo\vid2vid
pip install -r requirements.txt

run:

python app.py

If everything goes well press Ctrl+C to exit the current running.

[shinhochoi]

Thanks for your explaination of installation steps. I found some typing errors and let others recognize these for installation.

1. No module named 'triton' --> I don't know why, but it does not affect and do not concern this. I hope this to be clear.

2. python img2img/multi.py --input ../images/inputs --output-dir ../images/outputs
   ; change the path of " --output-dir " --> output

3. run (place source video in path" ./images/input/"): --> there is mis-typing. python vid2vid/main.py --input ..images/inputs/input.mp4 --output ../images/outputs/output.mp4 --> No works.

   When work in the directory of the "example" (temporal solving) I placeed input.mp4 in the "examples" directory and modified path like this " -input ./input.mp4 ".

[mikecarnohan]

Thanks @Spr-Peach!

Install StreamDiffusion (latest version, recommended):

pip install git+https://github.com/cumulo-autumn/StreamDiffusion.git@main#egg=streamdiffusion[tensorrt]

Results in (as of 5/31/2024): DEPRECATION: git+https://github.com/cumulo-autumn/StreamDiffusion.git@main#egg=streamdiffusion[tensorrt] contains an egg fragment with a non-PEP 508 name pip 25.0 will enforce this behaviour change. A possible replacement is to use the req @ url syntax, and remove the egg fragment. Discussion can be found at https://github.com/pypa/pip/issues/11617

DEPRECATION: antlr4-python3-runtime is being installed using the legacy 'setup.py install' method, because it does not have a 'pyproject.toml' and the 'wheel' package is not installed. pip 23.1 will enforce this behaviour change. A possible replacement is to enable the '--use-pep517' option. Discussion can be found at https://github.com/pypa/pip/issues/8559

DEPRECATION: fire is being installed using the legacy 'setup.py install' method, because it does not have a 'pyproject.toml' and the 'wheel' package is not installed. pip 23.1 will enforce this behaviour change. A possible replacement is to enable the '--use-pep517' option. Discussion can be found at https://github.com/pypa/pip/issues/8559

Though the last two notices can be fixed by running: python -m pip install --upgrade pip and rerunning the original command

[mikecarnohan]

Also, if people are running into issues installing 11.8 CUDA-compatible TensorRT tools at this step (or any related step):

python -m streamdiffusion.tools.install-tensorrt

They can try this install method...

After activating .venv in the StreamDiffusion directory:

cd streamdiffusion\.venv
scripts\activate.bat

Run:

python -m pip install tensorrt-cu11