RT-DETR

RT-DETR (Real-Time DEtection TRansformer) is the first real-time end-to-end object detector in the DETR line.

• Paper: Zhao, Lv, Xu, Wei, Wang, Dang, Liu and Chen (Baidu Inc. / Peking University Shenzhen), DETRs Beat YOLOs on Real-time Object Detection, CVPR 2024, arXiv:2304.08069.
• Core claim: the YOLO family’s speed advantage is partly an accounting artefact, because YOLO latencies are reported without the NMS post-processing they require; once NMS is priced in, a properly redesigned DETR wins on both axes.
• Efficient hybrid encoder: decouples intra-scale attention from cross-scale fusion instead of running dense attention over the concatenated multi-scale sequence.
• Uncertainty-minimal query selection: picks encoder features whose classification and localization predictions agree.
• Headline result: RT-DETR-R50 reaches 53.1 AP at 108 FPS and RT-DETR-R101 54.3 AP at 74 FPS on COCO val2017 (T4, TensorRT FP16), beating both the $L$/$X$ YOLO models and DINO-Deformable-DETR-R50 (50.9 AP at 5 FPS).

Motivation: what NMS actually costs

The paper’s first contribution is measurement rather than architecture. NMS has two thresholds — a confidence threshold that pre-filters boxes and an IoU threshold that governs suppression — and its runtime is dominated by how many boxes survive the first filter.

Counting the boxes remaining after confidence filtering on the same input shows the anchor-based YOLOv5 producing roughly $3\times$ more candidates than the anchor-free YOLOv8 at every threshold (≈9.3k vs ≈3.6k at $0.001$), which is why the paper concludes that anchor-free detectors outperform anchor-based ones at equivalent accuracy simply because they feed NMS less work.

The accuracy/latency trade-off that follows is sharp (Table 1, YOLOv8 on COCO val2017, T4 + TensorRT FP16, timing only the TensorRT EfficientNMS kernel):

IoU thr. (conf $=0.001$)	AP	NMS (ms)		Conf. thr. (IoU $=0.7$)	AP	NMS (ms)
0.5	52.1	2.24		0.001	52.9	2.36
0.6	52.6	2.29		0.01	52.4	1.73
0.8	52.8	2.46		0.05	51.2	1.06

Execution time rises as the confidence threshold falls or the IoU threshold rises: a low confidence threshold leaves more boxes to screen, a high IoU threshold discards fewer per round. Buying the last $1.7$ AP (51.2 → 52.9) more than doubles NMS time (1.06 → 2.36 ms), and the best-AP setting is also the slowest. Worse, the right setting is scenario-dependent — recall-oriented deployments want a low confidence and high IoU threshold, precision-oriented ones the reverse — so the thresholds are a deployment-time tuning burden that destabilizes both speed and accuracy. This motivates an end-to-end speed benchmark: measure a detector’s average inference time over COCO val2017 including the NMS plugin, excluding only I/O and MemoryCopy. Under that benchmark, YOLO latencies grow by 1–2.4 ms while a DETR’s do not move at all.

The complementary problem is that DETRs are not real-time. Deformable DETR made multi-scale features affordable, but the concatenated multi-scale sequence is long, and the encoder ends up consuming 49% of the GFLOPs while contributing only 11% of the AP. RT-DETR attacks exactly that block.

Architecture

RT-DETR is a backbone (ResNet-18/34/50/101 or a scaled CSPResNet), an efficient hybrid encoder, uncertainty-minimal query selection, and a standard DETR decoder with auxiliary prediction heads. The last three backbone stages $\{{\mathcal{S}}_3,{\mathcal{S}}_4,{\mathcal{S}}_5\}$ (strides 8/16/32) feed the encoder.

Efficient hybrid encoder

The design is justified by an explicit ladder of encoder variants, each a step away from the DINO-Deformable-R50 encoder:

• A — no encoder at all, features simply concatenated.
• B — a single-scale Transformer encoder (SSE) applied per level, then concatenated: intra-scale interaction only.
• C — concatenate first, then a multi-scale Transformer encoder (MSE): intra- and cross-scale interaction simultaneously, on one long sequence. This is the conventional DETR encoder.
• D — decouple them: SSE for intra-scale, a PANet-style convolutional structure (CSF) for cross-scale fusion.
• E — the proposed hybrid encoder: AIFI for intra-scale, CCFF for cross-scale.

The two modules:

AIFI (Attention-based Intra-scale Feature Interaction) applies a single-scale Transformer encoder layer to ${\mathcal{S}}_5$ only. The argument is semantic: self-attention on high-level features relates conceptual entities that low-level features do not yet contain, so intra-scale attention on ${\mathcal{S}}_3$/${\mathcal{S}}_4$ is not merely wasteful but confusing — it risks duplicating and tangling with the high-level interactions. Restricting it to ${\mathcal{S}}_5$ (variant ${\text{D}}_{{\mathcal{S}}_5}$ in the ablation) cuts latency by 35% and gains 0.4 AP, so the claim is empirically load-bearing rather than a pure efficiency compromise. It also removes the encoder’s dependence on the total sequence length ${\sum }_l{H}_l{W}_l$: attention now runs on the shortest map only.

CCFF (CNN-based Cross-scale Feature Fusion) is a PANet-style fusion path into which fusion blocks are inserted. A fusion block takes two adjacent-scale features, concatenates them, splits into two $1\times 1$-convolution paths to adjust channels, passes one through $N$ RepBlocks built from RepConv, and merges the two paths by element-wise add. Being convolutional, it is cheap where the feature maps are large.

The encoder is then, exactly:

$\mathcal{Q}=\mathcal{K}=\mathcal{V}=\mathrm{Flatten}({\mathcal{S}}_5)$

${\mathcal{F}}_5=\mathrm{Reshape}(\mathrm{AIFI}(\mathcal{Q},\mathcal{K},\mathcal{V}))$

$\mathcal{O}=\mathrm{CCFF}(\{{\mathcal{S}}_3,{\mathcal{S}}_4,{\mathcal{F}}_5\})$

where $\mathrm{Reshape}$ restores the flattened sequence to the spatial shape of ${\mathcal{S}}_5$.

Uncertainty-minimal query selection

Two-stage query selection (Deformable DETR, DINO) picks the top-$K$ encoder features by classification score to initialize object queries. RT-DETR’s objection: the detector must model both category and location, so the quality of a feature is a latent variable jointly correlated with classification and localization — ranking by classification alone selects features that are confidently classified but poorly localized, injecting epistemic uncertainty into the decoder’s initialization.

Define the uncertainty of an encoder feature $\hat{\mathcal{X}}\in {\mathbb{R}}^D$ as the discrepancy between its predicted localization distribution $\mathcal{P}$ and classification distribution $\mathcal{C}$:

$\mathcal{U}(\hat{\mathcal{X}})=\Vert \mathcal{P}(\hat{\mathcal{X}})-\mathcal{C}(\hat{\mathcal{X}})\Vert$

and fold it into the loss so that it is optimized, not merely measured:

$\mathcal{L}(\hat{\mathcal{X}},\hat{\mathcal{Y}},\mathcal{Y})={\mathcal{L}}_{box}(\hat{\mathbf{b}},\mathbf{b})+{\mathcal{L}}_{cls}(\mathcal{U}(\hat{\mathcal{X}}),\hat{\mathbf{c}},\mathbf{c})$

with $\hat{\mathcal{Y}}=\{\hat{\mathbf{c}},\hat{\mathbf{b}}\}$ the predicted category and box and $\mathcal{Y}$ the ground truth. Because the uncertainty term enters the classification loss, a feature whose box and class disagree is penalized in its score, so the top-$K$ ranking by score becomes a ranking by joint quality — this is the same IoU-aware alignment idea as the Task-Aligned-Assignment metric and the IoU-aware query selection of DINO, applied to query initialization rather than label assignment. Selection remains top-$K$ ($K=300$) by classification score; the boxes of the selected features become the initial position queries.

The scatterplot of the selected features (only those with classification score $>0.5$) is the evidence: uncertainty-minimal selection (purple) concentrates toward the top-right — high classification and high IoU — while vanilla selection (green) sits at the bottom-right. Quantitatively there are 138% more purple than green dots with classification score $>0.5$ and 120% more with both scores $>0.5$.

Flexible speed tuning

Because the DETR decoder is a stack of homogeneous layers each with its own auxiliary prediction head, any prefix of the stack is a complete detector. RT-DETR exploits this for deployment: train once with $D=6$ decoder layers, then choose how many to run at inference with no retraining and no fine-tuning, trading accuracy for latency along a continuum. The AP difference between adjacent layers shrinks with depth, so the top layers are nearly free to drop (Table 5, RT-DETR-R50, $6\times$ schedule):

Decoder layers used	AP	Latency (ms)
6	53.1	9.3
5	53.0	8.8
4	52.7	8.3
3	52.4	7.9
2	51.3	7.5
1	49.1	7.0

Dropping the last layer costs $0.1$ AP and saves $0.5$ ms. This is what a NMS-based detector cannot offer: its only latency knob is the NMS thresholds, which move accuracy and require re-validation. The scaled models exploit the same mechanism — RT-DETR-R50-Dec${}^k$ denotes $k$ decoder layers at inference.

Scaling

RT-DETR scales in two directions. Width via the embedding dimension and CCFF channel count, depth via the number of Transformer layers and RepBlocks; additionally the backbone is swapped (R18/R34/R50/R101 or CSPResNet) and the number of decoder layers is varied. The scaled variants halve the RepBlock channels and leave everything else unchanged.

Hyperparameters

Reported training configuration (Appendix Table A; RT-DETR-R50):

Setting	Value
Dataset	COCO train2017 / val2017 (+ Objects365 for pre-training)
Backbone	ImageNet-pretrained ResNet-50 (also R18/R34/R101)
Input size	$640\times 640$ (other DETRs use $800\times 1333$)
Optimizer	AdamW
Base LR	$1\times 10^{-4}$
Backbone LR	$1\times 10^{-5}$
Frozen BN	True
Warm-up	Linear, start factor $0.001$, 2000 steps
Weight decay	$1\times 10^{-4}$
Gradient clipping	Norm $0.1$
EMA decay	$0.9999$
AIFI layers	1
RepBlocks per fusion block	3
Embedding dim	256
Feedforward dim	1024
Attention heads	8
Feature scales	3 (${\mathcal{S}}_3,{\mathcal{S}}_4,{\mathcal{S}}_5$)
Decoder layers	6
Object queries	300
Decoder sampling points	4
Matching cost weights	class $2.0$ ($\alpha =0.25$, $\gamma =2.0$), bbox $5.0$, GIoU $2.0$
Loss weights	class $1.0$ ($\alpha =0.75$, $\gamma =2.0$), bbox $5.0$, GIoU $2.0$
Denoising	200 denoising queries, label noise ratio $0.5$, box noise scale $1.0$
Schedule	$1\times$ = 12 epochs; final results use $6\times$ = 72 epochs
Augmentation	random color distort, expand, crop, flip, resize
Hardware	4× NVIDIA Tesla V100, batch size 16

The decoder’s training recipe (denoising queries, box noise, auxiliary losses) follows DINO almost exactly — the denoising groups trace back to DN-DETR and the box-as-query formulation to DAB-DETR. RT-DETR’s novelty is confined to the encoder and the query-selection objective.

Results

COCO val2017, $L$/$X$-scale comparison; FPS at batch size 1 on a T4 GPU with TensorRT FP16, YOLO numbers measured under the end-to-end benchmark (Table 2):

Model	Backbone	Epochs	Params (M)	GFLOPs	FPS	AP	AP${}_{50}$	AP${}_S$	AP${}_L$
YOLOv5-L	–	300	46	109	54	49.0	67.3	–	–
PPYOLOE-L	–	300	52	110	94	51.4	68.9	31.4	66.1
YOLOv6-L (v3.0)	–	300	59	150	99	52.8	70.3	34.4	70.1
YOLOv7-L	–	300	36	104	55	51.2	69.7	35.2	66.7
YOLOv8-L	–	–	43	165	71	52.9	69.8	35.3	69.8
YOLOv8-X	–	–	68	257	50	53.9	71.0	35.7	70.7
Deformable-DETR	R50	50	40	173	–	46.2	65.2	28.8	61.7
DN-Deformable-DETR	R50	50	48	195	–	48.6	67.4	31.0	63.7
DINO-Deformable-DETR	R50	36	47	279	5	50.9	69.0	34.6	64.6
RT-DETR	R50	72	42	136	108	53.1	71.3	34.8	70.0
RT-DETR	R101	72	76	259	74	54.3	72.7	36.0	72.1

Against YOLOs, RT-DETR-R50 improves on YOLOv5-L / PP-YOLOE-L / YOLOv6-L by 4.1 / 1.7 / 0.3 AP while raising FPS by 100.0% / 14.9% / 9.1% and cutting parameters by 8.7% / 19.2% / 28.8%; against YOLOv7-L / YOLOv8-L it gains 1.9 / 0.2 AP at 96.4% / 52.1% more FPS. Against DETRs, RT-DETR-R50 beats DINO-Deformable-DETR-R50 by 2.2 AP and roughly $21\times$ the FPS (108 vs 5) — the FLOPs drop from 279G to 136G, half of which is the encoder redesign, the rest the $640\times 640$ input.

Hybrid encoder ablation (Table 3, RT-DETR-R50, $1\times$ schedule) — each row is one rung of the A→E ladder:

Variant	AP	Params (M)	Latency (ms)
A (no encoder)	43.0	31	7.2
B (+ intra-scale SSE)	44.9	32	11.1
C (+ cross-scale, MSE)	45.6	32	13.3
D (decoupled: SSE + CSF)	46.4	35	12.2
D${}_{{\mathcal{S}}_5}$ (intra-scale on ${\mathcal{S}}_5$ only)	46.8	35	7.9
E (AIFI + CCFF)	47.9	42	9.3

Reading the ladder: intra-scale interaction is worth $+1.9$ AP but $+54\%$ latency; cross-scale fusion is worth another $+0.7$ AP at $+20\%$ latency; decoupling the two buys $+0.8$ AP while reducing latency 8%, so it is not merely a cost saving; restricting intra-scale attention to ${\mathcal{S}}_5$ cuts latency 35% and still gains $0.4$ AP, confirming that low-level intra-scale attention is actively unhelpful; and the AIFI/CCFF refinement adds $+1.5$ AP, with parameters up 20% but latency down 24%.

Query selection ablation (Table 4, RT-DETR-R50, $1\times$). ${\text{Prop}}_{cls}$ and ${\text{Prop}}_{both}$ are the proportions of selected features with classification score $>0.5$, and with both classification and IoU $>0.5$:

Query selection	AP	${\text{Prop}}_{cls}$	${\text{Prop}}_{both}$
Vanilla	47.9	0.35	0.30
Uncertainty-minimal	48.7	0.82	0.67

Scaled models vs. $S$/$M$ YOLOs (Appendix Table B): RT-DETR-R50-Dec${}^2$ (154 FPS, 50.3 AP) through Dec${}^6$ (125 FPS, 52.2 AP) outperform every $M$ YOLO; RT-DETR-R18-Dec${}^3$ hits 46.5 AP at 217 FPS, beating all $S$ models. Against YOLOv6-M, RT-DETR-R50-Dec${}^5$ gains $0.9$ AP and 36% FPS; against YOLOv6-S, RT-DETR-R18-Dec${}^2$ gains $0.5$ AP and 18% FPS.

Objects365 pre-training (Appendix Table C): 12 epochs on Objects365 then fine-tuning on COCO (60 epochs for R18, 24 for R50/R101) gives $49.2$ / $55.3$ / $56.2$ AP for R18/R50/R101 — improvements of $+2.7$ / $+2.2$ / $+1.9$ AP.

Limitations

• Small objects remain the weak point, the same limitation as every other DETR at comparable model size: RT-DETR-R50’s AP${}_S$ is $0.5$ below YOLOv8-L’s and RT-DETR-R101’s is $0.9$ below YOLOv7-X’s, even though both beat those models on overall AP. Attention on ${\mathcal{S}}_5$ alone plus a purely convolutional path for ${\mathcal{S}}_3$/${\mathcal{S}}_4$ is a plausible cause the paper does not isolate.
• The speed claims depend on the paper’s own end-to-end benchmark; against the conventional (NMS-excluded) numbers that YOLO papers report, the margin narrows considerably. YOLOv10, which removes NMS from the YOLO side entirely, subsequently reversed several of these comparisons.
• The $640\times 640$ input is part of the speed story and is not disentangled from the encoder redesign in the headline FLOPs comparison against DINO-Deformable-DETR ($800\times 1333$).
• The uncertainty $\mathcal{U}$ is an $L_p$ discrepancy between two heads’ outputs with no calibration guarantee — it is an alignment regularizer that behaves like epistemic uncertainty, not a principled estimate of it.
• Training still takes 72 epochs on 4 V100s; RT-DETR is real-time at inference only.

See also

• DETR — the set-prediction formulation RT-DETR keeps, including the NMS-free bipartite matching that makes the speed argument possible
• Deformable DETR — supplies the multi-scale deformable decoder; its encoder is exactly the computational bottleneck RT-DETR removes
• DINO — RT-DETR’s decoder recipe and the IoU-aware query selection that uncertainty-minimal selection refines
• DN-DETR — the denoising training used in RT-DETR’s decoder (200 denoising queries)
• DAB-DETR — box-as-query formulation underlying the position queries
• Conditional DETR — an earlier attack on DETR’s query optimization difficulty
• YOLOv8 — the detector used throughout the NMS analysis, and a baseline in Table 2
• LW-DETR — the concurrent real-time DETR that drops the CNN backbone for a plain ViT; it re-measures RT-DETR in its own environment and beats R50 by 0.8 mAP at lower latency
• YOLOv10 — the reply from the YOLO side: NMS-free training via consistent dual assignments
• Task-Aligned-Assignment — the same classification/localization alignment idea, applied to label assignment instead of query selection
• Implementing Research Papers