YOLOv8-TensorRT
YOLOv8-TensorRT copied to clipboard
Published
20 hours ago •
triple-Mu
triple-Mu
ERROR when runing infer-det-without-torch.py
First of all, thank you very much for your work @triple-Mu . Let me describe the problem I encountered.
- Commit id: 6c396351551c3617286d3b3abac2d5b6d54a8833
- TensorRT version: 8.2.4.2
- Driver Version: 545.29.02
- CUDA Version: 12.3
Using infer_det.py for inference, no problem
CUDA_VISIBLE_DEVICES=1 python3 infer-det.py --engine yolov8n.engine --imgs data --out-dir outputs --device cuda:0
Then I use infer-det-without-torch.py to replace it
CUDA_VISIBLE_DEVICES=1 python3 infer-det-without-torch.py --engine yolov8n.engine --imgs data --out-dir outputs
Traceback (most recent call last):
File "infer-det-without-torch.py", line 87, in <module>
main(args)
File "infer-det-without-torch.py", line 40, in main
bboxes, scores, labels = det_postprocess(data)
File "/workdir/YOLOv8-TensorRT/models/utils.py", line 212, in det_postprocess
bboxes, scores, labels = bboxes[idx], scores[idx], labels[idx]
IndexError: arrays used as indices must be of integer (or boolean) type
I have reviewed the implementation inside and found that it is sent to idx=nms (bboxes, scores, iou_thres). However, the content returned by the nms provided by pytorch does not match the content returned by the custom nms:
...
...
idx = nms(bboxes, scores, iou_thres)
print(type(idx))
...
...
# CUDA_VISIBLE_DEVICES=1 python3 infer-det.py --engine yolov8n.engine --imgs data --out-dir outputs --device cuda:0
<class 'torch.Tensor'>
<class 'torch.Tensor'>
<class 'torch.Tensor'>
root@user-X11DAi-N:/workdir/YOLOv8-TensorRT#
root@user-X11DAi-N:/workdir/YOLOv8-TensorRT# CUDA_VISIBLE_DEVICES=1 python3 infer-det-without-torch.py --engine yolov8n.engine --imgs data --out-dir outputs
<class 'tuple'>
Traceback (most recent call last):
File "infer-det-without-torch.py", line 87, in <module>
main(args)
File "infer-det-without-torch.py", line 40, in main
bboxes, scores, labels = det_postprocess(data)
File "/workdir/YOLOv8-TensorRT/models/utils.py", line 213, in det_postprocess
bboxes, scores, labels = bboxes[idx], scores[idx], labels[idx]
IndexError: arrays used as indices must be of integer (or boolean) type
Can you provide your onnx model and an image for me to test?
- pt -> onnx
# python3 export-det.py --weights yolov8n.pt --iou-thres 0.65 --conf-thres 0.25 --topk 100 --opset 11 --sim --input-shape 1 3 640 640 --device cuda:0
YOLOv8n summary (fused): 168 layers, 3151904 parameters, 0 gradients, 8.7 GFLOPs
WARNING: The shape inference of TRT::EfficientNMS_TRT type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of TRT::EfficientNMS_TRT type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of TRT::EfficientNMS_TRT type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of TRT::EfficientNMS_TRT type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of TRT::EfficientNMS_TRT type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of TRT::EfficientNMS_TRT type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of TRT::EfficientNMS_TRT type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of TRT::EfficientNMS_TRT type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of TRT::EfficientNMS_TRT type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of TRT::EfficientNMS_TRT type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of TRT::EfficientNMS_TRT type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
WARNING: The shape inference of TRT::EfficientNMS_TRT type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function.
ONNX export success, saved as yolov8n.onnx
- onnx -> engine
# python3 build.py --weights yolov8n.onnx --iou-thres 0.65 --conf-thres 0.25 --topk 100 --fp16 --device cuda:0
[12/07/2023-19:43:34] [TRT] [W] parsers/onnx/onnx2trt_utils.cpp:364: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[12/07/2023-19:43:34] [TRT] [W] parsers/onnx/onnx2trt_utils.cpp:392: One or more weights outside the range of INT32 was clamped
[12/07/2023-19:43:34] [TRT] [W] input "images" with shape: (1, 3, 640, 640) dtype: DataType.FLOAT
[12/07/2023-19:43:34] [TRT] [W] output "num_dets" with shape: (1, 1) dtype: DataType.INT32
[12/07/2023-19:43:34] [TRT] [W] output "bboxes" with shape: (1, 100, 4) dtype: DataType.FLOAT
[12/07/2023-19:43:34] [TRT] [W] output "scores" with shape: (1, 100) dtype: DataType.FLOAT
[12/07/2023-19:43:34] [TRT] [W] output "labels" with shape: (1, 100) dtype: DataType.INT32
[12/07/2023-19:50:46] [TRT] [W] Build tensorrt engine finish.
Save in /workdir/YOLOv8-TensorRT/yolov8n.engine
- infer
root@user-X11DAi-N:/workdir/YOLOv8-TensorRT# python3 infer-det.py --engine yolov8n.engine --imgs data/zidane.jpg --out-dir outputs --device cuda:0
root@user-X11DAi-N:/workdir/YOLOv8-TensorRT#
root@user-X11DAi-N:/workdir/YOLOv8-TensorRT# python3 infer-det-without-torch.py --engine yolov8n.engine --imgs data/zidane.jpg --out-dir outputs
Traceback (most recent call last):
File "infer-det-without-torch.py", line 87, in <module>
main(args)
File "infer-det-without-torch.py", line 40, in main
bboxes, scores, labels = det_postprocess(data)
File "/workdir/YOLOv8-TensorRT/models/utils.py", line 212, in det_postprocess
bboxes, scores, labels = bboxes[idx], scores[idx], labels[idx]
IndexError: arrays used as indices must be of integer (or boolean) type
I encountered the same problem, there seems to be a little problem with the author's nms processing
Different nms functions are used in infer-det.py and infer-det-without-torch.py, so the model can run the latter's post-processing in the former and errors will occur. What I don't understand is why the latter's post-processing No information from labels is used. The nms function can be changed as follows：
YOLOv8-TensorRT/models/utils.py
def nms(boxes, scores, labels, iou_thresh, conf_thresh=0.1):
indices = np.where(scores >= conf_thresh)[0]
boxes = boxes[indices]
scores = scores[indices]
labels = labels[indices]
keep = []
order = scores.argsort()[::-1]
while order.size > 0:
i = order[0]
keep.append(i)
rest_boxes = boxes[order[1:]]
iou = compute_iou(boxes[i], rest_boxes)
indices = np.where(iou < iou_thresh)[0]
order = order[indices + 1]
keep_boxes = boxes[keep]
keep_scores = scores[keep]
keep_labels = labels[keep]
return keep_boxes, keep_scores, keep_labels
In addition
# bboxes, scores, labels = bboxes[idx], scores[idx], labels[idx]
replaced by
bboxes, scores, labels = idx
Could you please pr for this? Okay, I'll try my best, and I will add the preprocess and postprocess logic of the multi-batch reasoning part.
Hi @triple-Mu @minwim , the key of this problem is that nms-with-numpy is different with torchvision.ops.nms
so I try to implment a new one which is same with torchvision.ops.nms, I will push PR for this fix.
def nms(bboxes: ndarray, scores: ndarray, iou_thresh: float):
"""
Performs non-maximum suppression (NMS) on the boxes according
to their intersection-over-union (IoU).
NMS iteratively removes lower scoring boxes which have an
IoU greater than iou_threshold with another (higher scoring)
box.
If multiple boxes have the exact same score and satisfy the IoU
criterion with respect to a reference box, the selected box is
not guaranteed to be the same between CPU and GPU. This is similar
to the behavior of argsort in PyTorch when repeated values are present.
Args:
bboxes (ndarray[N, 4])): boxes to perform NMS on. They
are expected to be in ``(x1, y1, x2, y2)`` format with ``0 <= x1 < x2`` and
``0 <= y1 < y2``.
scores (ndarray[N]): scores for each one of the boxes
iou_thresh (float): discards all overlapping boxes with IoU > iou_threshold
Returns:
ndarray: int64 tensor with the indices of the elements that have been kept
by NMS, sorted in decreasing order of scores
"""
x1 = bboxes[:, 0]
y1 = bboxes[:, 1]
x2 = bboxes[:, 2]
y2 = bboxes[:, 3]
areas = (y2 - y1) * (x2 - x1)
result = []
index = scores.argsort()[::-1]
while index.size > 0:
i = index[0]
result.append(i)
x11 = np.maximum(x1[i], x1[index[1:]])
y11 = np.maximum(y1[i], y1[index[1:]])
x22 = np.minimum(x2[i], x2[index[1:]])
y22 = np.minimum(y2[i], y2[index[1:]])
w = np.maximum(0, x22 - x11 + 1)
h = np.maximum(0, y22 - y11 + 1)
overlaps = w * h
ious = overlaps / (areas[i] + areas[index[1:]] - overlaps)
idx = np.where(ious <= iou_thresh)[0]
index = index[idx + 1]
return np.array(result, dtype=int)