Enhance Quick Start

[RicoSuaveGuapo]

Hi, thanks for this interesting framework.

While I was trying to follow the readme under VideoRAG/VideoRAG-algorithm. I found myself missing some dependency or encountering some error. So I thought I could write a help-you issue to contribute to the open source community :)

I use

• RTX 3060 (5090 needs torch with cuda128, which seems cannot fit in this repo dependency)
• Ubuntu 24.04.3 LTS

Beside list of dependency list in here. There is an additional dependency need to installed.

pip install ollama==0.5.3

When I follow the torch version in readme, during query, I encounter the following

next_tokens = torch.multinomial(probs, num_samples=1).squeeze(1)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: probability tensor contains either `inf`, `nan` or element < 0

So I switched to the following torch version.

pip install torch==2.3.1 torchaudio==2.3.1 torchvision==0.18.1 --index-url https://download.pytorch.org/whl/cu118

But if you use these versions, during indexing, you will have

ModuleNotFoundError: No module named 'torchvision.transforms.functional_tensor'

By this hacking (yeah, very nasty), I was able to run the indexing.

I really hope we can have a file like requirements.txt or pyproject.toml to include all the dependencies.

If you like me, git does not support lfs and cannot use sudo. Use conda-forge to install git-lfs.

conda install -c conda-forge git-lfs

Then follow the readme to download the checkpoint.

After the env setup. You can start the indexing like readme here.

You might encounter the following error

  subaudio.write_audiofile(os.path.join(video_segment_cache_path, audio_file), codec='mp3', verbose=False, logger=None)
    ^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'write_audiofile'

This is because your video does not have audio. And yes, if you do not wish to change the source code, change to the video with audio.

If you encounter this error

Could not load library libcudnn_ops_infer.so.8. Error: libcudnn_ops_infer.so.8: cannot open shared object file: No such file or directory

Add the following to your indexing Python script

os.environ["LD_LIBRARY_PATH"] = (
    "{your_conda_env_path}/lib/python3.11/site-packages/nvidia/cublas/lib:"
    "{your_conda_env_path}/lib/python3.11/site-packages/nvidia/cudnn/lib"
)

for example of your_conda_env_path, is like /home/ricoli/anaconda3/envs/videorag_demo

My full indexing python file is like this

import os
from pathlib import Path
import logging
import warnings
import multiprocessing

warnings.filterwarnings("ignore")
logging.getLogger("httpx").setLevel(logging.WARNING)

from videorag._llm import openai_4o_mini_config
from videorag import VideoRAG, QueryParam

# os.environ["OPENAI_API_KEY"] = "" replace with your api key

# if you encounter the augmentation (which in torchvision) error when running the code, please refer to the link below for a solution:
# https://blog.csdn.net/lanxing147/article/details/136625264

os.environ["LD_LIBRARY_PATH"] = (
    "{your_conda_env_path}/lib/python3.11/site-packages/nvidia/cublas/lib:"
    "{your_conda_env_path}/lib/python3.11/site-packages/nvidia/cudnn/lib"
)

if __name__ == '__main__':
    multiprocessing.set_start_method('spawn')

    # Please enter your video file path in this list; there is no limit on the length.
    # Here is an example; you can use your own videos instead.
    video_paths = [
        'some_video.mp4',
    ]
    work_dir = Path('./videorag-workdir')
    work_dir.mkdir(parents=True, exist_ok=True)
    videorag = VideoRAG(llm=openai_4o_mini_config, working_dir=str(work_dir))
    videorag.insert_video(video_path_list=video_paths)

[Re-bin]

Hi 👋!

Many thanks for your contribution! I will add the relevant details and a link to this issue in the README as soon as possible.

Thanks again :)

Best regards, Xubin