BentoML
BentoML copied to clipboard
Published
3 weeks ago
•
bentoml
bentoml
Error when using torch_hub_yolov5 Custom Mode
Describe the bug
I inquired about the issue before ==> https://github.com/bentoml/BentoML/issues/2602
First of all, thank you so much for answering my inquiry and giving me a solution. https://github.com/bentoml/gallery/tree/main/custom_runner/torch_hub_yolov5
So I implemented your proposal. I checked that it went normally if I did as you suggested.
2022-08-04T11:18:51+0000 [WARNING] [cli] Using lowercased runnable class name 'yolov5runnable' for runner.
2022-08-04T11:18:52+0000 [INFO] [cli] Starting development BentoServer from "service.py:svc" running on http://127.0.0.1:3000 (Press CTRL+C to quit)
2022-08-04T11:18:53+0000 [WARNING] [dev_api_server] Using lowercased runnable class name 'yolov5runnable' for runner.
start=========================================
Downloading: "https://github.com/ultralytics/yolov5/archive/master.zip" to /root/.cache/torch/hub/master.zip
2022-08-04T11:18:57+0000 [INFO] [dev_api_server] YOLOv5 2022-8-4 Python-3.7.5 torch-1.11.0+cu102 CUDA:0 (Tesla P40, 24452MiB)
YOLOv5 2022-8-4 Python-3.7.5 torch-1.11.0+cu102 CUDA:0 (Tesla P40, 24452MiB)
2022-08-04T11:18:58+0000 [INFO] [dev_api_server] Downloading https://github.com/ultralytics/yolov5/releases/download/v6.1/yolov5s.pt to yolov5s.pt...
Downloading https://github.com/ultralytics/yolov5/releases/download/v6.1/yolov5s.pt to yolov5s.pt...
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 14.1M/14.1M [00:00<00:00, 22.3MB/s]
2022-08-04T11:18:59+0000 [INFO] [dev_api_server]
2022-08-04T11:19:03+0000 [INFO] [dev_api_server] Fusing layers...
Fusing layers...
2022-08-04T11:19:03+0000 [INFO] [dev_api_server] YOLOv5s summary: 213 layers, 7225885 parameters, 0 gradients
YOLOv5s summary: 213 layers, 7225885 parameters, 0 gradients
2022-08-04T11:19:03+0000 [INFO] [dev_api_server] Adding AutoShape...
Adding AutoShape...
-----------------------------------------------
GPU available
-----------------------------------------------
2022-08-04T11:19:03+0000 [INFO] [dev_api_server] Application startup complete.
Application startup complete.
But Error occurs when running with custom model.
I only changed the following from your service.py source. service .py
torch.hub.load('ultralytics/yolov5', 'yolov5s')
==> torch.hub.load("ultralytics/yolov5", 'custom', path='my_custom_yolov5_model.pt', force_reload=True)
Error occurs when using custom model as follows.
2022-08-04T11:20:33+0000 [WARNING] [cli] Using lowercased runnable class name 'yolov5runnable' for runner.
2022-08-04T11:20:33+0000 [INFO] [cli] Starting development BentoServer from "service.py:svc" running on http://127.0.0.1:3000 (Press CTRL+C to quit)
2022-08-04T11:20:35+0000 [WARNING] [dev_api_server] Using lowercased runnable class name 'yolov5runnable' for runner.
start=========================================
Downloading: "https://github.com/ultralytics/yolov5/archive/master.zip" to /root/.cache/torch/hub/master.zip
2022-08-04T11:20:37+0000 [INFO] [dev_api_server] YOLOv5 2022-8-4 Python-3.7.5 torch-1.11.0+cu102 CUDA:0 (Tesla P40, 24452MiB)
YOLOv5 2022-8-4 Python-3.7.5 torch-1.11.0+cu102 CUDA:0 (Tesla P40, 24452MiB)
2022-08-04T11:20:38+0000 [ERROR] [dev_api_server] Traceback (most recent call last):
File "/root/.cache/torch/hub/ultralytics_yolov5_master/hubconf.py", line 47, in _create
model = DetectMultiBackend(path, device=device, fuse=autoshape) # detection model
File "/root/.cache/torch/hub/ultralytics_yolov5_master/models/common.py", line 334, in __init__
model = attempt_load(weights if isinstance(weights, list) else w, device=device, inplace=True, fuse=fuse)
File "/root/.cache/torch/hub/ultralytics_yolov5_master/models/experimental.py", line 81, in attempt_load
ckpt = (ckpt.get('ema') or ckpt['model']).to(device).float() # FP32 model
File "/opt/venv/tf2.3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1186, in __getattr__
type(self).__name__, name))
AttributeError: 'DetectMultiBackend' object has no attribute 'get'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/root/.cache/torch/hub/ultralytics_yolov5_master/hubconf.py", line 51, in _create
model = attempt_load(path, device=device, fuse=False) # arbitrary model
File "/root/.cache/torch/hub/ultralytics_yolov5_master/models/experimental.py", line 81, in attempt_load
ckpt = (ckpt.get('ema') or ckpt['model']).to(device).float() # FP32 model
File "/opt/venv/tf2.3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1186, in __getattr__
type(self).__name__, name))
AttributeError: 'DetectMultiBackend' object has no attribute 'get'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/opt/venv/tf2.3/lib/python3.7/site-packages/starlette/routing.py", line 645, in lifespan
async with self.lifespan_context(app):
File "/opt/venv/tf2.3/lib/python3.7/site-packages/starlette/routing.py", line 540, in __aenter__
await self._router.startup()
File "/opt/venv/tf2.3/lib/python3.7/site-packages/starlette/routing.py", line 624, in startup
handler()
File "/opt/venv/tf2.3/lib/python3.7/site-packages/bentoml/_internal/runner/runner.py", line 226, in init_local
self._init_local()
File "/opt/venv/tf2.3/lib/python3.7/site-packages/bentoml/_internal/runner/runner.py", line 217, in _init_local
self._init(LocalRunnerRef)
File "/opt/venv/tf2.3/lib/python3.7/site-packages/bentoml/_internal/runner/runner.py", line 211, in _init
runner_handle = handle_class(self)
File "/opt/venv/tf2.3/lib/python3.7/site-packages/bentoml/_internal/runner/runner_handle/local.py", line 25, in __init__
self._runnable = runner.runnable_class(**runner.runnable_init_params) # type: ignore
File "/home/bentoml_pytorch/golf_service.py", line 14, in __init__
self.model = torch.hub.load("ultralytics/yolov5", 'custom', path='/home/yolov5_torchserve_v1/saved_model.pt', force_reload=True, autoshape=True)
File "/opt/venv/tf2.3/lib/python3.7/site-packages/torch/hub.py", line 404, in load
model = _load_local(repo_or_dir, model, *args, **kwargs)
File "/opt/venv/tf2.3/lib/python3.7/site-packages/torch/hub.py", line 433, in _load_local
model = entry(*args, **kwargs)
File "/root/.cache/torch/hub/ultralytics_yolov5_master/hubconf.py", line 74, in custom
return _create(path, autoshape=autoshape, verbose=_verbose, device=device)
File "/root/.cache/torch/hub/ultralytics_yolov5_master/hubconf.py", line 69, in _create
raise Exception(s) from e
Exception: 'DetectMultiBackend' object has no attribute 'get'. Cache may be out of date, try `force_reload=True` or see https://github.com/ultralytics/yolov5/issues/36 for help.
Traceback (most recent call last):
File "/root/.cache/torch/hub/ultralytics_yolov5_master/hubconf.py", line 47, in _create
model = DetectMultiBackend(path, device=device, fuse=autoshape) # detection model
File "/root/.cache/torch/hub/ultralytics_yolov5_master/models/common.py", line 334, in __init__
model = attempt_load(weights if isinstance(weights, list) else w, device=device, inplace=True, fuse=fuse)
File "/root/.cache/torch/hub/ultralytics_yolov5_master/models/experimental.py", line 81, in attempt_load
ckpt = (ckpt.get('ema') or ckpt['model']).to(device).float() # FP32 model
File "/opt/venv/tf2.3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1186, in __getattr__
type(self).__name__, name))
AttributeError: 'DetectMultiBackend' object has no attribute 'get'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/root/.cache/torch/hub/ultralytics_yolov5_master/hubconf.py", line 51, in _create
model = attempt_load(path, device=device, fuse=False) # arbitrary model
File "/root/.cache/torch/hub/ultralytics_yolov5_master/models/experimental.py", line 81, in attempt_load
ckpt = (ckpt.get('ema') or ckpt['model']).to(device).float() # FP32 model
File "/opt/venv/tf2.3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1186, in __getattr__
type(self).__name__, name))
AttributeError: 'DetectMultiBackend' object has no attribute 'get'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/opt/venv/tf2.3/lib/python3.7/site-packages/starlette/routing.py", line 645, in lifespan
async with self.lifespan_context(app):
File "/opt/venv/tf2.3/lib/python3.7/site-packages/starlette/routing.py", line 540, in __aenter__
await self._router.startup()
File "/opt/venv/tf2.3/lib/python3.7/site-packages/starlette/routing.py", line 624, in startup
handler()
File "/opt/venv/tf2.3/lib/python3.7/site-packages/bentoml/_internal/runner/runner.py", line 226, in init_local
self._init_local()
File "/opt/venv/tf2.3/lib/python3.7/site-packages/bentoml/_internal/runner/runner.py", line 217, in _init_local
self._init(LocalRunnerRef)
File "/opt/venv/tf2.3/lib/python3.7/site-packages/bentoml/_internal/runner/runner.py", line 211, in _init
runner_handle = handle_class(self)
File "/opt/venv/tf2.3/lib/python3.7/site-packages/bentoml/_internal/runner/runner_handle/local.py", line 25, in __init__
self._runnable = runner.runnable_class(**runner.runnable_init_params) # type: ignore
File "/home/bentoml_pytorch/golf_service.py", line 14, in __init__
self.model = torch.hub.load("ultralytics/yolov5", 'custom', path='/home/yolov5_torchserve_v1/saved_model.pt', force_reload=True, autoshape=True)
File "/opt/venv/tf2.3/lib/python3.7/site-packages/torch/hub.py", line 404, in load
model = _load_local(repo_or_dir, model, *args, **kwargs)
File "/opt/venv/tf2.3/lib/python3.7/site-packages/torch/hub.py", line 433, in _load_local
model = entry(*args, **kwargs)
File "/root/.cache/torch/hub/ultralytics_yolov5_master/hubconf.py", line 74, in custom
return _create(path, autoshape=autoshape, verbose=_verbose, device=device)
File "/root/.cache/torch/hub/ultralytics_yolov5_master/hubconf.py", line 69, in _create
raise Exception(s) from e
Exception: 'DetectMultiBackend' object has no attribute 'get'. Cache may be out of date, try `force_reload=True` or see https://github.com/ultralytics/yolov5/issues/36 for help.
Please review the error
For your information, I confirmed that my model(my_custom_yolov5_model.pt) is serving normally with flask.
To reproduce
Same as the guide https://github.com/bentoml/gallery/tree/main/custom_runner/torch_hub_yolov5
Expected behavior
Same as the guide https://github.com/bentoml/gallery/tree/main/custom_runner/torch_hub_yolov5
Environment
Same as the guide https://github.com/bentoml/gallery/tree/main/custom_runner/torch_hub_yolov5
@darkking-park does it work if you run torch.hub.load("ultralytics/yolov5", 'custom', path='my_custom_yolov5_model.pt', force_reload=True) in a python shell?
@parano The python flask is in use as below.
if __name__ == "__main__":
parser = argparse.ArgumentParser(description="Flask api exposing yolov5 model")
parser.add_argument("--port", default=5000, type=int, help="port number")
args = parser.parse_args()
model = torch.hub.load("ultralytics/yolov5", "custom", path='./model/best.pt', force_reload=True, autoshape=True)
model.eval()
app.run(host="0.0.0.0", port=args.port)
I want to use bento rather than flask
@darkking-park were you able to run the example with a brand new torch hub model downloaded from the git repo?
Where did you put the 'my_custom_yolov5_model.pt file? is it under the same directory?
@parano
were you able to run the example with a brand new torch hub model downloaded from the git repo?
https://github.com/ultralytics/yolov5
With the pretrained model in this github, I trained it myself and worked on it based on the github contents below.
https://github.com/ultralytics/yolov5/issues/36
Where did you put the 'my_custom_yolov5_model.pt file? is it under the same directory?
A model exists in the path set in torch.hub.load.
path='./model/best.pt'
@darkking-park it seems to me it's an issue with torchhub local cache or an issue with the model path and project path. It would be helpful if you could try setup everything from scratch and share us steps to reproduce, or I'd be happy to hop on a zoom call to help with debugging the issue, feel free to ping me in our slack channel if you'd like to do that.