[Bug] - mmengine - ERROR - ./mmdeploy/tools/deploy.py - create_process - 82 - visualize onnxruntime model failed. #2592
Description
Checklist
- I have searched related issues but cannot get the expected help.
- 2. I have read the FAQ documentation but cannot get the expected help.
- 3. The bug has not been fixed in the latest version.
Describe the bug
mmdetection-main
Reproduction
python ./mmdeploy/tools/deploy.py mmdeploy\configs\mmdet\detection\detection_onnxruntime_static.py configs/retinanet/retinanet_r50_fpn_ms-640-800-3x_coco.py checkpoints\retinanet_r50_fpn_mstrain_3x_coco_20210718_220633-88476508.pth demo\demo.jpg --test-img demo\demo.jpg --work-dir mmdeploy_model/re --show --dump-info
Environment
12/08 10:37:01 - mmengine - INFO -
12/08 10:37:01 - mmengine - INFO - **********Environmental information**********
12/08 10:37:16 - mmengine - INFO - sys.platform: win32
12/08 10:37:16 - mmengine - INFO - Python: 3.8.18 (default, Sep 11 2023, 13:39:12) [MSC v.1916 64 bit (AMD64)]
12/08 10:37:16 - mmengine - INFO - CUDA available: True
12/08 10:37:16 - mmengine - INFO - numpy_random_seed: 2147483648
12/08 10:37:16 - mmengine - INFO - GPU 0: NVIDIA GeForce GTX 1080 Ti
12/08 10:37:16 - mmengine - INFO - CUDA_HOME: C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.7
12/08 10:37:16 - mmengine - INFO - NVCC: Cuda compilation tools, release 11.7, V11.7.64
12/08 10:37:16 - mmengine - INFO - MSVC: 用于 x64 的 Microsoft (R) C/C++ 优化编译器 19.36.32537 版
12/08 10:37:16 - mmengine - INFO - GCC: n/a
12/08 10:37:16 - mmengine - INFO - PyTorch: 2.1.0
12/08 10:37:16 - mmengine - INFO - PyTorch compiling details: PyTorch built with:
- C++ Version: 199711
- MSVC 192930151
- Intel(R) Math Kernel Library Version 2020.0.2 Product Build 20200624 for Intel(R) 64 architecture applications
- Intel(R) MKL-DNN v3.1.1 (Git Hash 64f6bcbcbab628e96f33a62c3e975f8535a7bde4)
- OpenMP 2019
- LAPACK is enabled (usually provided by MKL)
- CPU capability usage: AVX2
- CUDA Runtime 12.1
- NVCC architecture flags: -gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,c
ode=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_90,code=sm_90
- CuDNN 8.8.1 (built against CUDA 12.0)
- Magma 2.5.4
- Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=12.1, CUDNN_VERSION=8.8.1, CXX_COMPILER=C:/cb/pytorch_1000000000000/work/tmp_bin/sccache-cl.exe, CXX_FLAGS=/DWIN32 /D_WINDOWS /GR /EHsc /bigobj /FS -DUSE_PTHREADPOOL -
DNDEBUG -DUSE_KINETO -DLIBKINETO_NOCUPTI -DLIBKINETO_NOROCTRACER -DUSE_FBGEMM -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE /utf-8 /wd4624 /wd4068 /wd4067 /wd4267 /wd4661 /wd4717 /wd4244 /wd4804 /wd4273, LAPACK_INFO=mkl, PERF_WITH_AV
X=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_DISABLE_GPU_ASSERTS=OFF, TORCH_VERSION=2.1.0, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=OFF, USE_NNPA
CK=OFF, USE_OPENMP=ON, USE_ROCM=OFF,
12/08 10:37:16 - mmengine - INFO - TorchVision: 0.16.0+cu121
12/08 10:37:16 - mmengine - INFO - OpenCV: 4.8.1
12/08 10:37:16 - mmengine - INFO - MMEngine: 0.9.1
12/08 10:37:16 - mmengine - INFO - MMCV: 2.1.0
12/08 10:37:16 - mmengine - INFO - MMCV Compiler: MSVC 192930148
12/08 10:37:16 - mmengine - INFO - MMCV CUDA Compiler: 12.1
12/08 10:37:16 - mmengine - INFO - MMDeploy: 1.0.0+
12/08 10:37:16 - mmengine - INFO -
12/08 10:37:16 - mmengine - INFO - **********Backend information**********
12/08 10:37:16 - mmengine - INFO - tensorrt: None
12/08 10:37:16 - mmengine - INFO - ONNXRuntime: 1.8.1
12/08 10:37:16 - mmengine - INFO - ONNXRuntime-gpu: None
12/08 10:37:16 - mmengine - INFO - ONNXRuntime custom ops: Available
12/08 10:37:16 - mmengine - INFO - pplnn: None
12/08 10:37:16 - mmengine - INFO - ncnn: None
12/08 10:37:16 - mmengine - INFO - snpe: None
12/08 10:37:16 - mmengine - INFO - openvino: None
12/08 10:37:16 - mmengine - INFO - torchscript: 2.1.0+cu121
12/08 10:37:16 - mmengine - INFO - torchscript custom ops: NotAvailable
12/08 10:37:16 - mmengine - INFO - rknn-toolkit: None
12/08 10:37:16 - mmengine - INFO - rknn-toolkit2: None
12/08 10:37:16 - mmengine - INFO - ascend: None
12/08 10:37:17 - mmengine - INFO - coreml: None
12/08 10:37:17 - mmengine - INFO - tvm: None
12/08 10:37:17 - mmengine - INFO - vacc: None
12/08 10:37:17 - mmengine - INFO -
12/08 10:37:17 - mmengine - INFO - **********Codebase information**********
12/08 10:37:17 - mmengine - INFO - mmdet: 3.1.0
12/08 10:37:17 - mmengine - INFO - mmseg: None
12/08 10:37:17 - mmengine - INFO - mmcls: None
12/08 10:37:17 - mmengine - INFO - mmocr: None
12/08 10:37:17 - mmengine - INFO - mmedit: None
12/08 10:37:17 - mmengine - INFO - mmdet3d: None
12/08 10:37:17 - mmengine - INFO - mmpose: None
12/08 10:37:17 - mmengine - INFO - mmrotate: None
12/08 10:37:17 - mmengine - INFO - mmaction: None
12/08 10:37:17 - mmengine - INFO - mmrazor: NoneError traceback
12/08 10:34:59 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "Codebases" registry tree. As a workaround, the current "Codebases" registry in "mmdeploy" is used to build instance. This may cause unexpected f
ailure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized.
12/08 10:34:59 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "mmdet_tasks" registry tree. As a workaround, the current "mmdet_tasks" registry in "mmdeploy" is used to build instance. This may cause unexpect
ed failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized.
12/08 10:35:03 - mmengine - INFO - Start pipeline mmdeploy.apis.pytorch2onnx.torch2onnx in subprocess
12/08 10:35:05 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "Codebases" registry tree. As a workaround, the current "Codebases" registry in "mmdeploy" is used to build instance. This may cause unexpected f
ailure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized.
12/08 10:35:05 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "mmdet_tasks" registry tree. As a workaround, the current "mmdet_tasks" registry in "mmdeploy" is used to build instance. This may cause unexpect
ed failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized.
Loads checkpoint by local backend from path: checkpoints\retinanet_r50_fpn_mstrain_3x_coco_20210718_220633-88476508.pth
12/08 10:35:05 - mmengine - WARNING - DeprecationWarning: get_onnx_config will be deprecated in the future.
12/08 10:35:05 - mmengine - INFO - Export PyTorch model to ONNX: mmdeploy_model/re\end2end.onnx.
E:\Anaconda3\envs\openmmlab\lib\site-packages\mmdeploy\codebase\mmdet\models\detectors\single_stage.py:84: TracerWarning: Iterating over a tensor might cause the trace to be incorrect. Passing a tensor of different shape won't change t
he number of iterations executed (and might lead to errors or silently give incorrect results).
img_shape = [int(val) for val in img_shape]
E:\Anaconda3\envs\openmmlab\lib\site-packages\mmdeploy\codebase\mmdet\models\detectors\single_stage.py:84: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of P
ython values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
img_shape = [int(val) for val in img_shape]
E:\Anaconda3\envs\openmmlab\lib\site-packages\mmdeploy\core\optimizers\function_marker.py:160: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values
, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
ys_shape = tuple(int(s) for s in ys.shape)
E:\Anaconda3\envs\openmmlab\lib\site-packages\mmdeploy\codebase\mmdet\models\dense_heads\base_dense_head.py:109: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flo
w of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
assert cls_score.size()[-2:] == bbox_pred.size()[-2:]
E:\Anaconda3\envs\openmmlab\lib\site-packages\mmdeploy\pytorch\functions\topk.py:28: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create te
nsors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
k = torch.tensor(k, device=input.device, dtype=torch.long)
E:\Anaconda3\envs\openmmlab\lib\site-packages\mmdeploy\codebase\mmdet\models\task_modules\coders\delta_xywh_bbox_coder.py:38: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record
the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
assert pred_bboxes.size(0) == bboxes.size(0)
E:\Anaconda3\envs\openmmlab\lib\site-packages\mmdeploy\codebase\mmdet\models\task_modules\coders\delta_xywh_bbox_coder.py:40: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record
the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
assert pred_bboxes.size(1) == bboxes.size(1)
E:\Anaconda3\envs\openmmlab\lib\site-packages\mmdeploy\mmcv\ops\nms.py:392: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value wil
l be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if not is_dynamic_batch(deploy_cfg) and batch_size == 1:
E:\Anaconda3\envs\openmmlab\lib\site-packages\mmdeploy\mmcv\ops\nms.py:307: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out
of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
iou_threshold = torch.tensor([iou_threshold], dtype=torch.float32)
E:\Anaconda3\envs\openmmlab\lib\site-packages\mmdeploy\mmcv\ops\nms.py:308: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out
of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
score_threshold = torch.tensor([score_threshold], dtype=torch.float32)
E:\Anaconda3\envs\openmmlab\lib\site-packages\mmdeploy\mmcv\ops\nms.py:44: TracerWarning: Converting a tensor to a Python float might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will b
e treated as a constant in the future. This means that the trace might not generalize to other inputs!
score_threshold = float(score_threshold)
E:\Anaconda3\envs\openmmlab\lib\site-packages\mmdeploy\mmcv\ops\nms.py:45: TracerWarning: Converting a tensor to a Python float might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will b
e treated as a constant in the future. This means that the trace might not generalize to other inputs!
iou_threshold = float(iou_threshold)
E:\Anaconda3\envs\openmmlab\lib\site-packages\mmcv\ops\nms.py:123: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be trea
ted as a constant in the future. This means that the trace might not generalize to other inputs!
assert boxes.size(1) == 4
E:\Anaconda3\envs\openmmlab\lib\site-packages\mmcv\ops\nms.py:124: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be trea
ted as a constant in the future. This means that the trace might not generalize to other inputs!
assert boxes.size(0) == scores.size(0)
E:\Anaconda3\envs\openmmlab\lib\site-packages\torch\onnx\symbolic_opset9.py:5856: UserWarning: Exporting aten::index operator of advanced indexing in opset 11 is achieved by combination of multiple ONNX operators, including Reshape, Tr
anspose, Concat, and Gather. If indices include negative values, the exported graph will produce incorrect results.
warnings.warn(
12/08 10:35:11 - mmengine - INFO - Execute onnx optimize passes.
12/08 10:35:11 - mmengine - WARNING - Can not optimize model, please build torchscipt extension.
More details: https://github.com/open-mmlab/mmdeploy/tree/1.x/docs/en/experimental/onnx_optimizer.md
12/08 10:35:13 - mmengine - INFO - Finish pipeline mmdeploy.apis.pytorch2onnx.torch2onnx
12/08 10:35:13 - mmengine - INFO - Start pipeline mmdeploy.apis.utils.utils.to_backend in main process
12/08 10:35:13 - mmengine - INFO - Finish pipeline mmdeploy.apis.utils.utils.to_backend
12/08 10:35:13 - mmengine - INFO - visualize onnxruntime model start.
12/08 10:35:19 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "Codebases" registry tree. As a workaround, the current "Codebases" registry in "mmdeploy" is used to build instance. This may cause unexpected f
ailure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized.
12/08 10:35:19 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "mmdet_tasks" registry tree. As a workaround, the current "mmdet_tasks" registry in "mmdeploy" is used to build instance. This may cause unexpect
ed failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized.
12/08 10:35:19 - mmengine - WARNING - Failed to search registry with scope "mmdet" in the "backend_detectors" registry tree. As a workaround, the current "backend_detectors" registry in "mmdeploy" is used to build instance. This may ca
use unexpected failure when running the built modules. Please check whether "mmdet" is a correct scope, or whether the registry is initialized.
12/08 10:35:19 - mmengine - INFO - Successfully loaded onnxruntime custom ops from E:\Anaconda3\envs\openmmlab\lib\site-packages\mmdeploy\lib\mmdeploy_onnxruntime_ops.dll
2023-12-08:10:35:21 - root - ERROR - only integer tensors of a single element can be converted to an index
Traceback (most recent call last):
File "E:\Anaconda3\envs\openmmlab\lib\site-packages\mmdeploy\utils\utils.py", line 41, in target_wrapper
result = target(*args, **kwargs)
File "E:\Anaconda3\envs\openmmlab\lib\site-packages\mmdeploy\apis\visualize.py", line 90, in visualize_model
task_processor.visualize(
File "E:\Anaconda3\envs\openmmlab\lib\site-packages\mmdeploy\codebase\base\task.py", line 330, in visualize
visualizer.add_datasample(
File "E:\Anaconda3\envs\openmmlab\lib\site-packages\mmengine\dist\utils.py", line 419, in wrapper
return func(*args, **kwargs)
File "E:\Anaconda3\envs\openmmlab\lib\site-packages\mmdet\visualization\local_visualizer.py", line 468, in add_datasample
pred_img_data = self._draw_instances(image, pred_instances,
File "E:\Anaconda3\envs\openmmlab\lib\site-packages\mmdet\visualization\local_visualizer.py", line 132, in _draw_instances
text_colors = [text_palette[label] for label in labels]
File "E:\Anaconda3\envs\openmmlab\lib\site-packages\mmdet\visualization\local_visualizer.py", line 132, in <listcomp>
text_colors = [text_palette[label] for label in labels]
TypeError: only integer tensors of a single element can be converted to an index
12/08 10:35:21 - mmengine - ERROR - ./mmdeploy/tools/deploy.py - create_process - 82 - visualize onnxruntime model failed.