YOLOv10 Reduces Edge Inference Latency by 40 Percent

Real-time object detection on edge hardware just hit a new baseline. YOLOv10 drops non-maximum suppression during training, cuts parameter count, and delivers a 40 percent latency reduction on Jetson Orin and Snapdragon 8 Gen 3 compared to YOLOv9-C. The model ships with native ONNX export and TensorRT-friendly operators, so the path from PyTorch to production silicon is shorter than ever.

What Changed Under the Hood

YOLOv10 replaces the traditional NMS post-process with a consistent dual-assignment training strategy. One-to-many heads provide dense supervision; one-to-one heads produce inference-ready predictions without NMS. The architecture also introduces large-kernel depth-wise convolutions, partial self-attention modules, and rank-guided block design. Together they shave 1.8B FLOPs off the large variant while gaining 1.4 AP on COCO val2017.

bash

pip install ultralytics==8.2.0
# Export to ONNX with dynamic batch
yolo export model=yolov10l.pt format=onnx opset=17 dynamic=True simplify=True

TensorRT Optimization Pipeline

TensorRT 10.0+ natively supports the new attention and large-kernel ops. Build an INT8 calibration cache using 500 representative frames from your target domain. Enable FP16 fallback for layers not yet quantized. The resulting engine runs end-to-end without plugin code.

python

import tensorrt as trt
builder = trt.Builder(trt.Logger())
config = builder.create_builder_config()
config.set_flag(trt.BuilderFlag.FP16)
config.set_flag(trt.BuilderFlag.INT8)
config.set_calibration_profile(calibrator)
engine = builder.build_serialized_network(network, config)

Model	Params (M)	FLOPs (B)	Latency (ms) Jetson Orin	Latency (ms) Snapdragon 8 Gen 3	COCO AP
YOLOv9-C	25.3	102.4	18.7	22.1	53.0
YOLOv10-L	24.6	81.2	11.2	13.4	54.4
YOLOv10-X	29.8	160.1	15.8	18.9	55.1

💡

TipUse trtexec --dumpProfile to verify that NMS layers are absent. Any remaining NonMaxSuppression node means the ONNX export kept the old head; re-export with ultralytics 8.2.0+.

ONNX Runtime Alternative

For x86 edge boxes or Windows on ARM, ONNX Runtime 1.18+ with the TensorRT EP or DirectML EP matches TensorRT native latency within 5 percent. Enable graph optimization level 99 and intra-op threading equal to physical cores. The session option `ort.SessionOptions().graph_optimization_level = ort.GraphOptimizationLevel.ORT_ENABLE_ALL` is mandatory.

python

import onnxruntime as ort
so = ort.SessionOptions()
so.graph_optimization_level = ort.GraphOptimizationLevel.ORT_ENABLE_ALL
so.intra_op_num_threads = 8
session = ort.InferenceSession('yolov10l.onnx', so, providers=['TensorrtExecutionProvider', 'CUDAExecutionProvider'])

"
Eliminating NMS at inference removes the last variable-latency stage in the pipeline, which is critical for hard real-time SLAM loops.
— Dr. Kai Chen, NVIDIA Robotics

Deployment Checklist

1. Export with dynamic batch and simplified graph. 2. Generate INT8 calibration set from production sensor data, not COCO. 3. Profile with `trtexec --warmUp=200 --duration=30` to capture steady-state latency. 4. Validate mAP drop stays under 0.5 points post-quantization. 5. Package engine + label map + preprocessing params as a versioned artifact in your CI/CD registry.

✦

⚠️

WarningYOLOv10-X exceeds 15 ms on Orin at 1280x1280 input. If your SLA is 10 ms, stick to YOLOv10-L at 640x640 or prune channels with NVIDIA TAO Toolkit.