DETR Family Comparison

A side-by-side comparison of the eight DETR-line detectors covered in this wiki — DETR, Deformable DETR, Conditional DETR, DAB-DETR, DN-DETR, DINO, RT-DETR and LW-DETR — in publication date, COCO AP, parameter count, FLOPs, throughput and training cost. Every number here is transcribed from the papers themselves (mostly from each paper’s own comparison table, which is why the same baseline rows recur with identical values across sources); nothing is re-measured. The individual articles carry the methods, the equations and the full hyperparameters.

Read the caveats before the tables

Cross-paper numbers in this family are not interchangeable, for six reasons:

1. Backbone and resolution. The canonical comparison point is ResNet-50 with the shorter side in $[480,800]$ and the longer side $\le 1333$. The two real-time models break this: RT-DETR and LW-DETR both train and evaluate at a fixed $640\times 640$, which is a large part of why their FLOPs are low. LW-DETR breaks the backbone convention too — it is the only model here with no CNN backbone at all, a plain ViT instead. DC5 variants ($16\times$-resolution) are a different operating point again, roughly doubling FLOPs for 1–2 AP.
2. Single-scale vs. multi-scale. DETR, Conditional DETR and DAB-DETR/DN-DETR in their base form use one feature map; Deformable DETR, DINO and RT-DETR use 3–5; LW-DETR uses one below large and two above. Multi-scale is worth several AP, almost all of it on small objects, and costs FLOPs. Comparing a single-scale model to a 4-scale model on AP alone is meaningless.
3. Schedule. AP figures range over 12, 24, 36, 50, 72 and 500 epochs. Convergence speed is the story of this line, so the epoch column is not a footnote — it is a primary axis.
4. Pre-training data. Everything through DINO’s ResNet-50 rows is COCO-only. RT-DETR’s headline numbers are COCO-only with an Objects365 variant reported separately; LW-DETR’s headline numbers are Objects365-pretrained throughout, worth $+5.5$ mAP on average by its own ablation. This is the single largest confounder in the table, and it is not a small correction — it is bigger than most of the architectural deltas being compared.
5. Latency hardware differs per paper. Deformable DETR reports FPS on a V100; DINO on an A100; RT-DETR and LW-DETR on a T4 with TensorRT FP16 at batch size 1. These numbers cannot be compared across papers. Only the within-table comparisons are valid — and RT-DETR vs. LW-DETR is the one genuine cross-paper latency comparison here, because LW-DETR re-measured RT-DETR in its own environment.
6. NMS accounting. Every model here is NMS-free by construction, so unlike the YOLO comparisons in RT-DETR and LW-DETR there is no hidden post-processing cost in any of these latencies.

On CPU latency: none of these eight papers reports one. All of them target server GPUs (a T4 at the small end), and none ships a CPU benchmark, a mobile benchmark, or a quantized/edge variant — so any CPU figure would be invention. Only the two real-time papers address deployment-time cost control at all: RT-DETR via its post-training decoder-depth knob (below), LW-DETR via a five-point scale family from 2.0 ms to 19.1 ms — a design-time choice, not a post-training one. If you need a transformer detector on CPU, the honest statement is that this literature does not answer the question; the nearest analogue in the wiki is MobileSAM, which shows what the corresponding exercise (distil the heavy encoder, keep the head) looks like for a segmentation model.

Timeline and lineage

Model	arXiv	Venue	The one thing it changed	Builds on
DETR	May 2020	ECCV 2020	Detection as set prediction: Hungarian matching, no anchors, no NMS	—
Deformable DETR	Oct 2020	ICLR 2021	Sparse deformable attention over $K$ sampled points; multi-scale becomes affordable	DETR
Conditional DETR	Aug 2021	ICCV 2021	Conditional spatial query in decoder cross-attention; keeps dense attention	DETR
DAB-DETR	Jan 2022	ICLR 2022	The query is a 4D anchor box $(x,y,w,h)$, refined layer by layer	Conditional DETR
DN-DETR	Mar 2022	CVPR 2022	Denoising training: noised GT boxes as auxiliary queries, bypassing matching	DAB-DETR
DINO	Mar 2022	ICLR 2023	Contrastive denoising + mixed query selection + look forward twice	DN-DETR + Deformable DETR
RT-DETR	Apr 2023	CVPR 2024	Efficient hybrid encoder (attention on the top level only) → real-time	DINO lineage
LW-DETR	Jun 2024	arXiv	Plain ViT as the encoder, no CNN backbone; window-major memory layout; 3-layer decoder; Objects365 pre-training	DINO + Deformable + RT-DETR

Two independent branches leave DETR — sparse attention (Deformable) and better queries (Conditional → DAB → DN) — and DINO is where they merge. RT-DETR then re-engineers the merged design for latency rather than accuracy, and LW-DETR does the same job again from the opposite end: throw the CNN backbone away entirely, make a plain ViT cheap enough with window attention, and lean on large-scale pre-training for the accuracy.

The main table: ResNet-50, COCO val2017

Each row is the model’s own headline ResNet-50 configuration. FPS hardware is given per row precisely because it is not comparable across rows.

Model	Scales	Epochs	AP	AP${}_S$	AP${}_M$	AP${}_L$	Params	GFLOPs	FPS (hardware)
DETR-R50	1	500	42.0	20.5	45.8	61.1	41M	86	28 (V100)
DETR-DC5-R50	1	500	43.3	22.5	47.3	61.1	41M	187	12 (V100)
Deformable DETR-R50	4	50	43.8	26.4	47.1	58.0	40M	173	19 (V100)
Deformable DETR ++ two-stage	4	50	46.2	28.8	49.2	61.7	40M	173	19 (V100)
Conditional DETR-R50	1	50	40.9	20.8	44.6	59.2	44M	90	—
Conditional DETR-DC5-R50	1	50	43.8	24.0	47.6	60.7	44M	195	—
DAB-DETR-R50	1	50	42.2	21.5	45.7	60.3	44M	94	—
DAB-DETR-DC5-R50${}^{\ast }$	1	50	45.7	26.1	49.4	63.1	44M	216	—
DN-DETR-R50	1	50	44.1	22.9	48.0	63.4	44M	94	—
DN-DETR-DC5-R50	1	50	46.3	26.7	50.0	64.3	44M	202	—
DN-Deformable-DETR-R50	4	50	48.6	31.0	52.0	63.7	48M	195	—
DINO-4scale	4	36	50.9	34.6	54.1	64.6	47M	279	24 (A100)
DINO-5scale	5	36	51.2	35.0	54.3	65.3	47M	860	10 (A100)
RT-DETR-R50	3	72	53.1	34.8	—	70.0	42M	136	108 (T4, TRT FP16)
RT-DETR-R101	3	72	54.3	36.0	—	72.1	76M	259	74 (T4, TRT FP16)
LW-DETR-large${}^{†}$	2	25	56.1	—	—	—	47M	72	113 (T4, TRT FP16)
LW-DETR-R50${}^{†}$	1	25	54.4	—	—	—	55M	68	~115 (T4, TRT FP16)

${}^{†}$ Objects365-pretrained (LW-DETR reports no COCO-only headline). LW-DETR-large’s encoder is a plain ViT, not a ResNet; the R50 row is the paper’s own convolutional-encoder ablation, included as the nearest like-for-like backbone. LW-DETR reports no AP${}_S$/AP${}_M$/AP${}_L$ breakdown on COCO at all.

RT-DETR’s AP${}_L$ (70.0) is not comparable to the others’ — it is measured at $640\times 640$, and the paper’s own table places it against YOLOs rather than against the $800\times 1333$ DETRs. The same caveat applies to both LW-DETR rows, with the extra confound that their AP includes the Objects365 gain.

What the table says

• Parameters barely move. From DETR to DINO the count goes 41M → 47M, a 15% increase across three years and roughly +9 AP, and LW-DETR-large lands at 47M again. This family does not buy accuracy with parameters; it buys it with better optimization (denoising, anchor queries), better use of resolution (multi-scale, sparse attention) and — by 2024 — more pre-training data. Parameter count is close to useless as a discriminator here.
• FLOPs, by contrast, move a lot — 86G → 279G (DINO-4scale) → 860G (DINO-5scale) — and are the real cost axis. The two real-time models reverse the trend hard: RT-DETR-R50 at 136G, LW-DETR-large at 72G, roughly a quarter of DINO-4scale for +5.2 AP. Roughly half of RT-DETR’s saving is the encoder redesign and half the smaller input; LW-DETR’s comes from window attention, a 3-layer decoder and at most two feature scales.
• Small objects are where the family gained. AP${}_S$ goes 20.5 (DETR) → 26.4 (Deformable, multi-scale) → 31.0 (DN-Deformable) → 35.0 (DINO-5scale). DETR’s weakest metric improved by 70%, while AP${}_L$ went 61.1 → 65.3, a mere +7%. Essentially the whole story of the line, measured on COCO, is fixing small objects.

Convergence: AP under a short schedule

The clearest way to see what denoising and anchor queries actually bought is the 12-epoch ($1\times$) column, from DINO’s Table 1:

Model	12 epochs	50 epochs (or best long schedule)
DETR-DC5	15.5	43.3 (at 500 epochs)
Deformable DETR (4scale)	41.1	46.2
DAB-DETR-DC5	38.0	45.7
DN-Deformable-DETR (4scale)	43.4	48.6
DINO-4scale	49.0	50.9 (36 ep)
DINO-5scale	49.4	51.2 (36 ep)

DETR at 12 epochs is 15.5 AP — effectively a non-detector. DINO at 12 epochs beats DETR’s 500-epoch result by 6 AP. That is a ~40× reduction in schedule length for a better model, and it is why the 2020-era complaint “DETRs are impractical to train” no longer holds.

DN-DETR’s contribution isolates cleanly, since it is a training-only change with identical architecture, parameters and FLOPs to DAB-DETR:

Backbone	DAB-DETR	DN-DETR	Δ
R50	42.2	44.1	+1.9
R50-DC5	44.5	46.3	+1.8
R101	43.5	45.2	+1.7
R101-DC5	45.8	47.3	+1.5

Same 94 GFLOPs, same 44M parameters, zero inference cost — the denoising branch is discarded at test time.

Throughput and deployment

Within RT-DETR’s own benchmark (T4, TensorRT FP16, batch 1, end-to-end), the only place where a DETR-family latency comparison is apples-to-apples:

Model	AP	FPS
DINO-Deformable-DETR-R50	50.9	5
RT-DETR-R50	53.1	108

$+2.2$ AP at roughly 21× the throughput. That gap is the entire justification for the RT-DETR paper, and it comes from three places: the $640\times 640$ input, the encoder redesign (attention on ${\mathcal{S}}_5$ only), and dropping the 5th feature scale.

RT-DETR is also the only model in the family with a post-training latency knob. Because every decoder layer has its own auxiliary head, any prefix of the stack is a complete detector, so you pick the depth at inference with no retraining:

Decoder layers	6	5	4	3	2	1
AP	53.1	53.0	52.7	52.4	51.3	49.1
Latency (ms)	9.3	8.8	8.3	7.9	7.5	7.0

Dropping the last layer costs 0.1 AP and saves 0.5 ms. Every other model in this table has one deployment configuration.

LW-DETR’s Table 4 is the only place where two DETR-family real-time models are timed in the same environment — the authors re-measured RT-DETR on their own T4 rather than quoting its paper — so this is the one valid cross-paper latency row in the wiki. Both sides are Objects365-pretrained:

Model	Params	GFLOPs	Latency (ms)	AP
RT-DETR-R18	20M	30.0	4.6	49.2
RT-DETR-R50	42M	69.4	9.3	55.3
RT-DETR-R101	76M	131.0	13.5	56.2
LW-DETR-small	15M	16.6	2.9	48.0
LW-DETR-medium	28M	42.8	5.6	52.5
LW-DETR-large	47M	71.6	8.8	56.1
LW-DETR-xlarge	118M	174.2	19.1	58.3

LW-DETR-large beats RT-DETR-R50 by 0.8 AP while being marginally faster, and is within 0.1 AP of RT-DETR-R101 at two-thirds the latency. The xlarge point is where the family stops paying: 2.5× the parameters of large for +2.2 AP.

Training cost

From DINO’s Table 5 (8× A100, ResNet-50) — the axis that actually decides whether you can reproduce a paper on your own hardware:

Model	Time/epoch	Epochs	GPU memory	AP reached
DETR	~16 min	300	26 GB	41.2
Deformable DETR	~55 min	50	—	45.4
DINO	~55 min	12	16 GB	47.9

DINO costs the same per epoch as Deformable DETR and needs a quarter of the epochs — roughly 11 GPU-hours on 8×A100 for 47.9 AP, against ~46 for Deformable DETR’s 45.4 and ~80 for DETR’s 41.2. Deformable DETR’s own paper reports 325 GPU-hours against DETR’s 2000 for the 500-epoch schedule (V100), the same ratio measured differently.

The exception is the leaderboard result: DINO’s 63.3 AP test-dev needs SwinL plus Objects365 pre-training on 64 A100s for 26 epochs, which is not a reproducible number on a modest budget. RT-DETR’s 72-epoch schedule on 4 V100s is comparatively cheap, but it is still 6× DINO’s $1\times$ schedule — real-time refers to inference only.

LW-DETR is the expensive outlier on this axis and the tables above hide it: its COCO schedule is short (50 epochs for T/S/M, 25 for L/X, 180K iterations) but it sits on 30 epochs of Objects365 detection pre-training plus a CAEv2 MIM run on the ViT before that, and every headline number in the paper includes it. Without pre-training LW-DETR-large drops from 56.1 to 49.5 AP — below DINO-4scale. Reproducing LW-DETR from scratch is therefore a substantially larger job than reproducing DINO, even though its COCO fine-tune is the shortest schedule in the family.

Scaling to the top of the leaderboard

Only DINO pushes the family to maximum accuracy, with SwinL + Objects365 pre-training:

Model	Params	val2017 AP	test-dev AP	End-to-end
SwinV2-G	3.0B	61.9	63.1
Florence-CoSwin-H	> 637M	—	62.4
DINO-SwinL	218M	63.2	63.3	✓

The parameter column is the interesting one: DINO matches and beats SwinV2-G with $1/15$ the parameters, and was the first end-to-end detector to top COCO at all.

Which one to actually use

• Understanding the ideas — read DETR, then Deformable DETR and DAB-DETR. Everything after that is a refinement of one of those three ideas.
• Best accuracy on a normal budget, training from scratch — DINO with ResNet-50, 12 or 24 epochs. It is still the strongest accuracy-per-training-hour point in the family by a wide margin, because it does not assume a pre-training run you have to pay for.
• Deploying on a GPU with a latency target, from released weights — LW-DETR if you can use its Objects365-pretrained checkpoints, since that is where most of its accuracy lives; RT-DETR otherwise, and use its decoder-depth knob rather than re-tuning anything. Below ~5 ms LW-DETR is the only option in the family at all.
• Adding to an existing DETR you already train — DN-DETR’s denoising is the cheapest upgrade in the family: architecture-neutral, no inference cost, +1.5–2 AP, and it transfers to Deformable DETR, Anchor DETR, Faster R-CNN and Mask2Former.
• A small domain-specific dataset — LW-DETR is the only model here with published evidence on this: it leads on RF100 (100 small datasets, 7 domains) and on cross-domain UVO, with the margin largest in the domains least like COCO.
• CPU or mobile — none of these. The literature stops at server GPUs.

See also

• DETR, Deformable DETR, Conditional DETR, DAB-DETR, DN-DETR, DINO, RT-DETR, LW-DETR — the eight articles compared here
• YOLOv10 — the convergent evolution from the other direction: a YOLO that removes NMS, the property DETRs had from day one
• Task-Aligned-Assignment — how the YOLO line solves the assignment problem that Hungarian matching solves here
• GPUs for Local ML — what the hardware in the training-cost table actually costs
• Implementing Research Papers