h2o-llmstudio
h2o-llmstudio copied to clipboard
Add Dockerfile.
arno@rippa:/nfs4/llm/h2o-llmstudio(docker)$ docker build -t h2o-llmstudio .
arno@rippa:/nfs4/llm/h2o-llmstudio(docker)$ docker run --runtime=nvidia --shm-size=64g -p 10101:10101 --rm h2o-llmstudio
2023/04/23 04:55:13 #
2023/04/23 04:55:13 # ┌────────────────┐ H2O Wave
2023/04/23 04:55:13 # │ ┐┌┐┐┌─┐┌ ┌┌─┐ │ nightly 20230412224258
2023/04/23 04:55:13 # │ └┘└┘└─└└─┘└── │ © 2021 H2O.ai, Inc.
2023/04/23 04:55:13 # └────────────────┘
2023/04/23 04:55:13 # ┌──────────────────────────────────────┐
2023/04/23 04:55:13 # │ Running at http://localhost:10101/ │
2023/04/23 04:55:13 # └──────────────────────────────────────┘
2023/04/23 04:55:13 # {"address":":10101","base-url":"/","t":"listen","web-dir":"/root/.local/share/virtualenvs/workspace-dqq3IVyd/www"}
INFO: Will watch for changes in these directories: ['/workspace']
INFO: Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)
INFO: Started reloader process [1] using StatReload
/root/.local/share/virtualenvs/workspace-dqq3IVyd/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('/usr/local/nvidia/lib'), PosixPath('/usr/local/nvidia/lib64')}
warn(msg)
/root/.local/share/virtualenvs/workspace-dqq3IVyd/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: /usr/local/nvidia/lib:/usr/local/nvidia/lib64 did not contain libcudart.so as expected! Searching further paths...
warn(msg)
/root/.local/share/virtualenvs/workspace-dqq3IVyd/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('http'), PosixPath('//127.0.0.1'), PosixPath('8000')}
warn(msg)
INFO: Started server process [15]
INFO: Waiting for application startup.
2023/04/23 04:55:16 # {"host":"http://127.0.0.1:8000","route":"/","t":"app_add"}
2023-04-23 04:55:16,706 - INFO: STARTING APP
INFO: Application startup complete.
