INT8 Quantization and compiling for Google Coral

[dbro]

In case it is helpful to people, here is a repo that provides a version of YOLO v9 that can run on the Google Coral EdgeTPU device. https://github.com/dbro/frigate-detector-edgetpu-yolo9

There are two Colab notebooks that demonstrate (1) how to change the model to use ReLU6 as the activation function and then run fine-tuning, and (2) how to convert the .pt file into ONNX and then convert+quantize that to TFLite.

(1) https://colab.research.google.com/drive/1RAZqCHuDWFdFxqFpL78OVfZQvnGoM6rH?usp=sharing

(2) https://colab.research.google.com/drive/1AdFfPn1zYj86e_n2Bq3hvvNbIlyn8Vei?usp=sharing

Related issues:
#388
#327

[abhib1990]

Hello @dbro and team
Thank you for providing the solution for tflite conversion.
I am using your code to convert tflite in the same way that you mentioned like using RELU6
I could convert upto onnx and can see detections with 1 class.
But when I try to convert to tflite, getting error
ValueError: axes don't match array
Version I am using below
PyTorch version: 2.9.0+cu128
TensorFlow version: 2.19.0
ONNX version: 1.19.1
onnx2tf version: 1.28.3
Below are some logs

================================================================================
STEP 8: Exporting to ONNX...

Using ONNX opset: 18
Model has 2 outputs
Model final node has shape torch.Size([22, 64, 68, 68])
[torch.onnx] Obtain model graph for DetectionModel([...] with torch.export.export(..., strict=False)...
[torch.onnx] Obtain model graph for DetectionModel([...] with torch.export.export(..., strict=False)... ✅
[torch.onnx] Run decomposition...
[torch.onnx] Run decomposition... ✅
[torch.onnx] Translate the graph into ONNX...
[torch.onnx] Translate the graph into ONNX... ✅
✓ ONNX export successful: runs/train/yolov9-s-relu6/weights/best.onnx

Simplifying ONNX model...
✓ ONNX simplification successful

🔍 Verifying ONNX export:
Outputs: ['output0', 'output1']
output0: [1, 5376, 64]
output1: [1, 5376, 1]
Output 0: (1, 5376, 64)
Output 1: (1, 5376, 1)

================================================================================
STEP 8.5: Verifying ONNX output with Abbey Road image...

Original image size: (512, 512)
shapes of boxes_raw : (1, 5376, 64) , scores_raw : (1, 5376, 1)
Boxes shape (5376, 4)
Scores shape (5376, 1)
Boxes min -252.3391 max 771.3701
Scores min 0.0180 max 0.7935

✅ Score Precision Check:
Unique values: 153
Unique max values: 153

Found 37 detections above 0.25 confidence.
Found 5 detections after NMS.
Detection 15: person (conf=0.793) [291.2, 195.5, 335.9, 420.5]
Detection 28: person (conf=0.666) [209.3, 165.0, 282.1, 471.5]
Detection 29: person (conf=0.655) [103.8, 189.8, 195.6, 506.6]
Detection 3: person (conf=0.627) [470.1, 145.2, 512.5, 273.0]
Detection 24: person (conf=0.369) [437.2, 146.2, 534.7, 500.3]

✓ Detections saved to 'detections_onnx_full.jpg'

================================================================================
STEP 10: Preparing calibration data...

Found 22 images in Dataset
Loading 10 images...
Processed 10/10 images
Successfully loaded 10 images
Generating 10 random noise images...

✓ Created calibration data: calibration_images.npy
Shape: (20, 512, 512, 3) (NHWC)
Dtype: float32

🔍 Calibration data verification:
Shape: (20, 512, 512, 3)
Range: [0.0, 255.0]
Mean: 117.5
First image stats - min: 4.0, max: 255.0
Last image stats - min: 0.0, max: 254.0

================================================================================
STEPS 11 & 12: Converting and Quantizing with TFLiteConverter

**Step 1: Converting ONNX to TensorFlow SavedModel...
�[31mERROR:�[0m The trace log is below.
Traceback (most recent call last):
File "/mnt/workingDir/yolov9_tflite_support_git/yolov9/relu6_tflite/lib/python3.10/site-packages/onnx2tf/ops/Conv.py", line 69, in make_node
if graph_node.i(1).op != 'DequantizeLinear'
File "/mnt/workingDir/yolov9_tflite_support_git/yolov9/relu6_tflite/lib/python3.10/site-packages/onnx_graphsurgeon/ir/node.py", line 93, in i
return self.inputs[tensor_idx].inputs[producer_idx]
IndexError: list index out of range

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/mnt/workingDir/yolov9_tflite_support_git/yolov9/relu6_tflite/lib/python3.10/site-packages/onnx2tf/utils/common_functions.py", line 314, in print_wrapper_func
result = func(*args, **kwargs)
File "/mnt/workingDir/yolov9_tflite_support_git/yolov9/relu6_tflite/lib/python3.10/site-packages/onnx2tf/utils/common_functions.py", line 388, in inverted_operation_enable_disable_wrapper_func
result = func(*args, **kwargs)
File "/mnt/workingDir/yolov9_tflite_support_git/yolov9/relu6_tflite/lib/python3.10/site-packages/onnx2tf/utils/common_functions.py", line 57, in get_replacement_parameter_wrapper_func
func(*args, **kwargs)
File "/mnt/workingDir/yolov9_tflite_support_git/yolov9/relu6_tflite/lib/python3.10/site-packages/onnx2tf/ops/Conv.py", line 77, in make_node
input_weights = get_weights_constant_or_variable(
File "/mnt/workingDir/yolov9_tflite_support_git/yolov9/relu6_tflite/lib/python3.10/site-packages/onnx2tf/utils/common_functions.py", line 440, in auto_cast_wrapper_func
const_or_var = func(*args, **kwargs)
File "/mnt/workingDir/yolov9_tflite_support_git/yolov9/relu6_tflite/lib/python3.10/site-packages/onnx2tf/utils/common_functions.py", line 536, in get_weights_constant_or_variable
values = values.transpose(convertion_table)
ValueError: axes don't match array

�[31mERROR:�[0m input_onnx_file_path: runs/train/yolov9-s-relu6/weights/best.onnx
�[31mERROR:�[0m onnx_op_name: node_conv2d
�[31mERROR:�[0m Read this and deal with it. https://github.com/PINTO0309/onnx2tf#parameter-replacement
�[31mERROR:�[0m Alternatively, if the input OP has a dynamic dimension, use the -b or -ois option to rewrite it to a static shape and try again.
�[31mERROR:�[0m If the input OP of ONNX before conversion is NHWC or an irregular channel arrangement other than NCHW, use the -kt or -kat option.
�[31mERROR:�[0m Also, for models that include NonMaxSuppression in the post-processing, try the -onwdt option.**

Do you know this error?

[dbro]

I am not an expert (and not on any "team" by the way)

Maybe somewhere the dimension size of "1" is causing problems. Maybe try with more than one class?

[abhib1990]

Great Thank you for response.
I will try that out and come back to you.