DETR

DETR (DEtection TRansformer)

DETR is an object detection model introduced by Facebook AI Research in End-to-End Object Detection with Transformers (Carion, Massa, Synnaeve, Usunier, Kirillov, Zagoruyko; ECCV 2020, arXiv:2005.12872). It reformulates object detection as a direct set prediction problem, removing the hand-designed components — anchor boxes, region proposals, non-maximum suppression (NMS) — that earlier detectors (Faster R-CNN, YOLO, RetinaNet) relied on.

Core idea

Traditional detectors predict a large number of candidate boxes and then use surrogate tasks (anchor assignment, NMS) to reduce them to a final set. DETR instead:

1. Predicts a fixed-size set of N detections in parallel (e.g., N = 100), independent of image content.
2. Trains with a loss that performs bipartite matching between predictions and ground-truth objects, so each prediction is responsible for at most one object (or “no object”).

This makes the pipeline end-to-end differentiable with no post-processing step required at inference beyond a simple confidence threshold.

Architecture

DETR combines a CNN backbone with a standard Transformer encoder-decoder:

• CNN backbone (e.g., ResNet-50) extracts a lower-resolution feature map from the input image.
• Flatten + positional encoding: the feature map is flattened into a sequence of tokens and combined with fixed sinusoidal positional encodings (since the transformer itself is permutation-invariant).
• Transformer encoder: standard multi-head self-attention layers process the flattened image features, letting each location attend to all others — useful for reasoning about relationships between distant objects.
• Transformer decoder: takes N learned object queries (fixed, learned embeddings, one per “slot”) and attends to the encoder output via cross-attention, transforming each query into an output embedding. Unlike autoregressive transformers, all N queries are decoded in parallel.
• Prediction heads: each decoder output embedding is passed through a shared feed-forward network (FFN) to predict a class label (including a special “no object” / ∅ class) and a bounding box (center x, y, width, height, normalized).

Bipartite matching loss

Because the N predictions are unordered, DETR needs a way to assign each ground-truth object to exactly one prediction before computing a loss. It finds the optimal bipartite matching σ between predictions and ground truth (padded with ∅ to size N) that minimizes a total matching cost, solved efficiently with the Hungarian algorithm.

The matching cost per pair combines:

• Class prediction cost (negative predicted probability of the correct class)
• Bounding box cost, using a mix of L1 loss and generalized IoU (GIoU) loss (L1 alone is scale-dependent and penalizes large boxes disproportionately)

Once matched, the training loss is the sum of:

• Negative log-likelihood for class prediction (with down-weighted loss for the ∅ class, since most predictions are unmatched)
• L1 + GIoU loss on box coordinates, only for matched pairs

This is the key trick that lets DETR train without NMS: since each ground-truth object is matched to a unique prediction, the model is never rewarded for producing duplicate boxes for the same object.

Auxiliary losses

Since predictions are also produced at intermediate decoder layers, DETR applies the same matching + loss at every decoder layer (auxiliary losses), which speeds up convergence and improves accuracy.

Results and characteristics

• On COCO, DETR achieves accuracy comparable to a well-tuned Faster R-CNN baseline, with a simpler pipeline.
• Strengths: much better performance on large objects (global self-attention helps reason about the whole image); simpler architecture with no hand-crafted anchors or NMS; the encoder attention maps are often interpretable, separating individual object instances.
• Weaknesses: notably worse performance on small objects; very slow to train — the original model required 500 epochs to converge, largely because the bipartite matching assignment is unstable early in training; quadratic complexity of self-attention over image feature maps is expensive for high-resolution inputs.
• DETR also generalizes naturally to panoptic segmentation by adding a mask head on top of the decoder outputs.

Hyperparameters

As reported in the original paper for the baseline COCO models:

Architecture

• Backbone: ResNet-50 or ResNet-101 (ImageNet-pretrained), dilated-C5 variant (DC5) also explored for higher-resolution features
• Transformer: 6 encoder layers, 6 decoder layers
• Hidden dimension (d_model): 256
• Attention heads: 8
• Feed-forward network dimension: 2048
• Dropout: 0.1 (applied in transformer layers)
• Number of object queries (N): 100

Optimization

• Optimizer: AdamW
• Learning rate (transformer): 1e-4
• Learning rate (backbone): 1e-5
• Weight decay: 1e-4
• Batch size: 64 images (across 16 GPUs, 4 images/GPU)
• Training length: 300 epochs (500 epochs in the original/slower schedule), with learning rate dropped by 10× after 200 epochs (or 400 for the 500-epoch schedule)
• Gradient clipping: max norm 0.1

Loss weights

• Classification loss weight: 1
• L1 bounding box loss weight (λ_L1): 5
• GIoU loss weight (λ_giou): 2
• No-object (∅) class weight: 0.1 (down-weighted to counteract class imbalance, since most of the N slots are unmatched)

Data augmentation

• Scale augmentation: resize so the shortest side is between 480 and 800 pixels, longest side ≤ 1333
• Random crop augmentation
• Random horizontal flip

Influence

DETR’s slow convergence and weak small-object performance motivated a wave of follow-up work, most notably Deformable DETR, which replaces full self/cross-attention with deformable (sparse, learned-offset) attention over a small set of key sampling points, drastically speeding up training and improving small-object detection. Later variants (DINO, DAB-DETR, DN-DETR, Conditional DETR, etc.) further refined query design and training stability. DETR is generally credited with establishing the transformer-based, NMS-free “set prediction” paradigm as a viable alternative to the anchor-based detection pipeline that had dominated the field since Faster R-CNN.

See also

• INDEX