Exit without error when using SD model

[ba2512005]

Expected Behavior

The application should load properly.

Current Behavior

It seems when SD is enabled, it loads the SD model and fails to load the other normal LLM model and also fails to load the webui and it exits without error.

Steps to Reproduce

1. Load the lollms webui initially
2. Install SD service and model and restart lollms webui
3. Restart lollms webui and it tries to load the SD model and the model picked from modelzoo

Possible Solution

I think it's failing because the GPU RAM is being overutilized on one gpu and not being split on the other GPU. Even with the device map to auto.

Context

running on a windows 11 system with ryzen 9 3950x, 128GB ram, rtx 3060 12GB and nvidia telsa p40

C:\Users\ba251\lollms-webui\scripts\windows>win_run.bat Starting LOLLMS Web UI... " ___ ___ ___ ___ ___ ___ " " /_\ /\ \ /_\ /_\ /_\ /\ \ " " /:/ / /::\ \ /:/ / /:/ / /::| | /::\ \ " " /:/ / /:/:\ \ /:/ / /:/ / /:|:| | /:/\ \ \ " " /:/ / /:/ :\ \ /:/ / /:/ / /:/|:|| :~\ \ \ " "/:// /:// :_\ /:// /:// /:/ |::::_\ /\ :\ \ _\ " ":\ \ :\ \ /:/ / :\ \ :\ \ //~~/:/ / :\ :\ // " " :\ \ :\ /:/ / :\ \ :\ \ /:/ / :\ :_\ " " :\ \ :/:/ / :\ \ :\ \ /:/ / :/:/ / " " :_\ ::/ / :_\ :_\ /:/ / ::/ / " " // // // // // // " By ParisNeo Trying to use Configuration at :global_paths_cfg.yaml global_paths_cfg.yaml found! Execution path : C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui Local zoos folder found ----------------------Paths information----------------------- personal_path:C:\Users\ba251\Documents\lollms personal_configuration_path:C:\Users\ba251\Documents\lollms\configs personal_databases_path:C:\Users\ba251\Documents\lollms\databases personal_models_path:C:\Users\ba251\Documents\lollms\models personal_user_infos_path:C:\Users\ba251\Documents\lollms\user_infos personal_trainers_path:C:\Users\ba251\Documents\lollms\trainers personal_trainers_path:C:\Users\ba251\Documents\lollms\trainers\gptqlora personal_data_path:C:\Users\ba251\Documents\lollms\data custom_personalities_path:C:\Users\ba251\Documents\lollms\custom_personalities custom_voices_path:C:\Users\ba251\Documents\lollms\custom_voices

----------------------Zoo information----------------------- bindings_zoo_path:C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\bindings_zoo personalities_zoo_path:C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\personalities_zoo extensions_zoo_path:C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\extensions_zoo models_zoo_path:C:\Users\ba251\lollms-webui\scripts\windows\lollms-webui\zoos\models_zoo

Configuration file is very old. Replacing with default configuration {'version': 58, 'binding_name': None, 'model_name': None, 'headless_server_mode': False, 'allowed_origins': [], 'host': 'localhost', 'port': 9600, 'discussion_prompt_separator': '!@>', 'seed': -1, 'n_predict': 1024, 'ctx_size': 4084, 'min_n_predict': 512, 'temperature': 0.9, 'top_k': 50, 'top_p': 0.95, 'repeat_last_n': 40, 'repeat_penalty': 1.2, 'n_threads': 8, 'personalities': ['generic/lollms'], 'active_personality_id': 0, 'override_personality_model_parameters': False, 'extensions': [], 'user_name': 'user', 'user_description': '', 'use_user_name_in_discussions': False, 'user_avatar': 'default_user.svg', 'use_user_informations_in_discussion': False, 'db_path': 'database.db', 'debug': False, 'debug_log_file_path': '', 'auto_update': True, 'auto_sync_personalities': True, 'auto_sync_extensions': True, 'auto_sync_bindings': True, 'auto_sync_models': True, 'auto_save': True, 'auto_title': False, 'hardware_mode': 'nvidia-tensorcores', 'auto_show_browser': True, 'copy_to_clipboard_add_all_details': False, 'enable_voice_service': False, 'xtts_base_url': 'http://127.0.0.1:8020', 'auto_read': False, 'current_voice': None, 'current_language': 'en', 'enable_sd_service': False, 'sd_base_url': 'http://127.0.0.1:7860', 'enable_ollama_service': False, 'ollama_base_url': 'http://0.0.0.0:11434', 'enable_petals_service': False, 'petals_base_url': 'http://0.0.0.0:8010', 'enable_lollms_service': False, 'lollms_base_url': 'http://0.0.0.0:1234', 'elastic_search_service': False, 'elastic_search_url': 'http://0.0.0.0:9200', 'vllm_service': False, 'vllm_url': 'http://0.0.0.0:8000', 'vllm_model_path': 'mistralai/Mistral-7B-v0.1', 'media_on': False, 'audio_in_language': 'en-US', 'auto_speak': False, 'audio_out_voice': None, 'audio_pitch': 1, 'audio_auto_send_input': True, 'audio_silenceTimer': 5000, 'activate_ltm': False, 'summerize_discussion': False, 'max_summary_size': 512, 'data_vectorization_visualize_on_vectorization': False, 'use_files': True, 'data_vectorization_activate': True, 'data_vectorization_method': 'tfidf_vectorizer', 'data_visualization_method': 'PCA', 'data_vectorization_save_db': False, 'data_vectorization_chunk_size': 512, 'data_vectorization_overlap_size': 128, 'data_vectorization_nb_chunks': 2, 'data_vectorization_build_keys_words': False, 'data_vectorization_force_first_chunk': False, 'data_vectorization_make_persistance': False, 'activate_internet_search': False, 'internet_vectorization_chunk_size': 512, 'internet_vectorization_overlap_size': 128, 'internet_vectorization_nb_chunks': 2, 'internet_nb_search_pages': 3, 'internet_quick_search': False, 'internet_activate_search_decision': False, 'pdf_latex_path': None, 'positive_boost': None, 'negative_boost': None, 'force_output_language_to_be': None, 'fun_mode': False, 'show_code_of_conduct': True, 'activate_audio_infos': True, 'whisper_model': 'base'} Added entries : ['headless_server_mode', 'allowed_origins'], removed entries:[]

---

| | | | | | | | | | | |
| || | | || || | | | | |__ | | | ||||||||||_____|___/ |_____| Forked from Auto1111's Stable diffusion api Integration in lollms by ParisNeo using mix1009's sdwebuiapi Loading lollms_sd Running on windows venv "C:\Users\ba251\Documents\lollms\shared\auto_sd\venv\Scripts\Python.exe"

INCOMPATIBLE PYTHON VERSION

This program is tested with 3.10.6 Python, but you have 3.11.7. If you encounter an error with "RuntimeError: Couldn't install torch." message, or any other error regarding unsuccessful package (library) installation, please downgrade (or upgrade) to the latest version of 3.10 Python and delete current Python and "venv" folder in WebUI's directory.

You can download 3.10 Python from here: https://www.python.org/downloads/release/python-3106/

Alternatively, use a binary release of WebUI: https://github.com/AUTOMATIC1111/stable-diffusion-webui/releases

Use --skip-python-version-check to suppress this warning.

fatal: No names found, cannot describe anything. Python 3.11.7 | packaged by Anaconda, Inc. | (main, Dec 15 2023, 18:05:47) [MSC v.1916 64 bit (AMD64)] Version: 1.7.0 Commit hash: a94cbe85eea026af1f268e01c1ca3325868b1f99 Launching Web UI with arguments: --api no module 'xformers'. Processing without... no module 'xformers'. Processing without... No module 'xformers'. Proceeding without it. Style database not found: C:\Users\ba251\Documents\lollms\shared\auto_sd\styles.csv Loading weights [6ce0161689] from C:\Users\ba251\Documents\lollms\shared\auto_sd\models\Stable-diffusion\v1-5-pruned-emaonly.safetensors Creating model from config: C:\Users\ba251\Documents\lollms\shared\auto_sd\configs\v1-inference.yaml Running on local URL: http://127.0.0.1:7860

To create a public link, set share=True in launch(). Startup time: 14.5s (prepare environment: 3.1s, import torch: 4.1s, import gradio: 1.9s, setup paths: 1.4s, initialize shared: 0.3s, other imports: 1.1s, setup codeformer: 0.2s, load scripts: 1.3s, create ui: 0.5s, gradio launch: 0.4s, add APIs: 0.1s). Service is available. Lollms_core found in the app space. Pulling last lollms_core remote: Enumerating objects: 14, done. remote: Counting objects: 100% (14/14), done. remote: Compressing objects: 100% (5/5), done. remote: Total 9 (delta 6), reused 7 (delta 4), pack-reused 0 Unpacking objects: 100% (9/9), 840 bytes | 15.00 KiB/s, done. From https://github.com/ParisNeo/lollms 002102b..909ec70 main -> origin/main Updating 002102b..909ec70 Fast-forward configs/config.yaml | 5 ++++- lollms/configs/config.yaml | 5 ++++- lollms/paths.py | 1 + 3 files changed, 9 insertions(+), 2 deletions(-) safe_store_path found in the app space. Pulling last safe_store_path Already up to date. Bindings zoo found in your personal space. Pulling last bindings zoo Applying attention optimization: Doggettx... done. Already up to date. Personalities zoo found in your personal space. Pulling last personalities zoo Model loaded in 3.5s (load weights from disk: 0.7s, create model: 0.6s, apply weights to model: 1.9s, calculate empty prompt: 0.1s). error: The following untracked working tree files would be overwritten by merge: thinking_methodologies/mixture_of_experts_perspective_ai/assets/logo.png Please move or remove them before you merge. Aborting Updating 46dafae..e425cd4 Extensions zoo found in your personal space. Pulling last Extensions zoo Already up to date. Models zoo found in your personal space. Pulling last Models zoo Already up to date.

Loading binding bs_exllamav2. Please wait ... Binding bs_exllamav2 loaded successfully. Loading model Code-290k-13B-exl2. Please wait ... Building model Code-290k-13B-exl2 Destroying model ------- Cuda VRAM usage ------- {'nb_gpus': 2, 'gpu_0_total_vram': 25769803776, 'gpu_0_used_vram': 8388608, 'gpu_0_model': 'Tesla P40', 'gpu_1_total_vram': 12884901888, 'gpu_1_used_vram': 3987734528, 'gpu_1_model': 'NVIDIA GeForce RTX 3060'} Cleared cache ------- Cuda VRAM usage ------- {'nb_gpus': 2, 'gpu_0_total_vram': 25769803776, 'gpu_0_used_vram': 8388608, 'gpu_0_model': 'Tesla P40', 'gpu_1_total_vram': 12884901888, 'gpu_1_used_vram': 3987734528, 'gpu_1_model': 'NVIDIA GeForce RTX 3060'} Creating model C:\Users\ba251\Documents\lollms\models\exl2\Code-290k-13B-exl2 Using device map: auto Loading model: C:/Users/ba251/Documents/lollms/models/exl2/Code-290k-13B-exl2 Press any key to continue . . .

Screenshots

If applicable, add screenshots to help explain the issue.

[ParisNeo]

Hi. I think I need to upgrade the automatic 1111's sd. I should separate its environment from lollms as they don't use the same python version anymore.

By the way. You can install it on your machine independantly and run it with the API server activated then lollms can use it directly. you can also use a remote version of SD, just set the url for the server in the artbot or lollms personality maker parameters.