SOLOv2

SOLOv2 is a box-free instance segmentation architecture that refines the original SOLO (“Segmenting Objects by LOcations”) formulation, which reduces instance segmentation to two per-pixel classification subtasks solvable by a plain FCN.

• Paper: Xinlong Wang, Rufeng Zhang, Tao Kong, Lei Li and Chunhua Shen, SOLOv2: Dynamic and Fast Instance Segmentation, NeurIPS 2020, arXiv:2003.10152.
• Dynamic convolution: replaces SOLO’s memory-heavy dense mask tensor with a mask kernel branch and a mask feature branch, combined by convolution at inference time.
• Matrix NMS: a new NMS that makes mask NMS parallel rather than sequential.
• Headline result: Res-50-FPN SOLOv2 reaches 38.8% mask AP at 18 FPS on COCO test-dev, outperforming SOLO by 1.9 AP while being 33% faster.
• Box byproduct: the bounding box of the predicted mask, with no detection training at all, reaches 44.9 box AP, ahead of many purpose-built detectors.

Core idea: segment objects by grid location

SOLO’s original insight, which SOLOv2 keeps: an instance can be identified purely by which grid cell its center falls into, with no anchor boxes, no region proposals, and no embedding-based grouping. The input image is conceptually divided into an $S\times S$ grid; if an object’s center lands in cell $(i,j)$, that cell is responsible for predicting a binary mask for that entire object. This gives $S^2$ possible instance “slots” per FPN level, output as an $S^2$-channel mask tensor $M\in {\mathbb{R}}^{H\times W\times S^2}$, where channel $k=i\cdot S+j$ contains the mask for whichever instance (if any) is centered in cell $(i,j)$:

$M_{i,j}=G_{i,j}\ast F$

where $F\in {\mathbb{R}}^{H\times W\times E}$ is a single level of backbone/FPN features and $G_{i,j}\in {\mathbb{R}}^{1\times 1\times E}$ is, in the original SOLO, simply the corresponding slice of a normal convolution’s learned, static weight tensor — i.e. the final layer of an ordinary FCN, with $S^2$ output channels standing in for $S^2$ per-location mask predictors.

Why SOLO needed fixing

Three bottlenecks in this direct formulation motivate SOLOv2:

1. Inefficient mask representation. Outputting the full $M\in {\mathbb{R}}^{H\times W\times S^2}$ tensor is expensive in both memory and compute, and — because $S$ differs per FPN level — the final layer producing it cannot be shared across levels, which wastes training signal. Most of $S^2$ is empty on any given image anyway (objects are sparse), so densely predicting and storing all $S^2$ channels is wasteful redundancy.
2. Inaccurate mask predictions. Finer boundary detail needs higher mask resolution, which further inflates the cost of a dense $S^2$-channel output.
3. Slow mask NMS. Computing pairwise mask IoU for standard sequential NMS is much more expensive for masks than for boxes, and the sequential dependency is the same latency bottleneck YOLACT’s Fast NMS also targets.

Dynamic convolution: decoupling kernel and feature

Fig. 2 (paper): SOLO’s single static final conv layer (top) is replaced by two decoupled branches (bottom) — a kernel branch predicting $S\times S$ dynamic convolution kernels $G$, and a feature branch producing one shared mask feature map $F$ — convolved together ($⊛$) only for the grid cells that actually contain an instance.

SOLOv2’s central change: separate what SOLO’s single conv layer conflated. Instead of one static layer whose weights double as “the $S^2$ mask predictor,” predict the convolution kernel $G$ and the feature map $F$ it will be convolved with as two independent outputs of two branches, then perform the convolution $M=G\ast F$ only at inference/training time, for however many grid cells actually contain an object center:

Mask kernel branch

Given a backbone+FPN feature map, resized to $S\times S\times C$ (via CoordConv — concatenating two extra channels of pixel coordinates normalized to $[-1,1]$, which the paper finds important for location-conditioned prediction), four $3\times 3$ convs followed by a final $3\times 3\times D$ conv produce the kernel tensor $G\in {\mathbb{R}}^{S\times S\times D}$. $D$ is the number of parameters of the dynamic conv kernel each grid cell predicts: $D=E$ for a $1\times 1$ dynamic conv, $D=9E$ for $3\times 3$ (the paper finds $1\times 1$ performs equivalently to $3\times 3$ and uses it by default with $E=256$ input channels). No activation function is applied to $G$‘s output — these are raw convolution weights, generated fresh per image, per grid cell, conditioned on that cell’s local features.

Mask feature branch

Rather than predicting mask features per-FPN-level as SOLO effectively did, SOLOv2 predicts one unified, high-resolution mask feature representation shared across the whole image: each FPN level $P_2$–$P_5$ is processed by repeated $3\times 3$ conv + group norm + ReLU + $2\times$ bilinear upsample stages (with $P_5$‘s deepest level additionally fed normalized pixel coordinates before its convolutions, matching the kernel branch’s positional conditioning), fused by element-wise sum into a single $F\in {\mathbb{R}}^{H\times W\times E}$ output at $1/4$ input resolution, followed by one more $1\times 1$ conv + group norm + ReLU. An ablation confirms the unified representation beats predicting mask features separately per level, especially for medium/large objects — in the separate scheme, large objects get assigned to low-resolution feature-pyramid levels, giving them coarser boundary prediction than they need.

Forming the mask and Matrix NMS

For each grid cell $(i,j)$ with a high enough classification confidence, its predicted kernel $G_{i,j,:}\in {\mathbb{R}}^D$ is convolved with the shared feature map $F$ to produce that instance’s mask — the dynamic part of “dynamic convolution”: the kernel is generated per-image and per-location rather than being a fixed, learned weight. This decoupling means the kernel branch only has to output $S\times S\times D$ values (small) instead of the full dense mask tensor, and the convolution itself is only ever computed for grid cells that survive a confidence threshold (0.1) — most of the $S^2$ locations are skipped entirely at inference.

Matrix NMS replaces the sequential IoU-then-remove loop of standard/soft-NMS with a parallel formulation motivated by Soft-NMS’s score-decay idea. For a prediction $m_j$, its suppression is modeled as depending on (a) the confidence penalty from any higher-scoring prediction $m_i$ that overlaps it, and (b) the probability that $m_i$ itself would already have been suppressed by something even higher-scoring — approximated by that most-overlapped higher-scoring prediction’s own decay:

$f({\text{iou}}_{\cdot ,i})={\min }_{\forall s_k>s_i}f({\text{iou}}_{k,i})deca{y}_j={\min }_{\forall s_i>s_j}\frac{f({\text{iou}}_{i,j})}{f({\text{iou}}_{\cdot ,i})}$

with $s_j=s_j\cdot deca{y}_j$ the updated score, and $f$ either linear ($1-\text{iou}$) or gaussian ($\exp (-{\text{iou}}^2/\sigma )$). Because both the pairwise IoU matrix and these column-wise min/max reductions are ordinary matrix operations, Matrix NMS needs no recursion and no sequential score sorting loop — it processes 500 masks in under 1 ms in plain Python, 9× faster than the paper’s own traditional-NMS baseline and 0.4 AP better than YOLACT’s Fast NMS.

Loss

$L=L_{cate}+\lambda L_{mask}$

$L_{cate}$ is Focal Loss for the per-grid-cell semantic category classification (does this cell contain an instance center, and of which class); $L_{mask}$ is Dice Loss on the assembled mask, following SOLO.

Hyperparameters

• Training: synchronous SGD, 8 GPUs, 16 images/batch total, 36 epochs (3×) by default, initial LR 0.01 divided by 10 at epochs 27 and 33; scale jitter with the shorter image side sampled from $[640,800]$.
• Kernel shape: $1\times 1$ conv, 256 input channels (found stable beyond 256; equivalent to $3\times 3$).
• Mask feature branch: fuses $P_2$–$P_5$, unified representation, final output at $1/4$ resolution.
• Real-time variants: speed-priority (2 conv layers in the prediction head, 448px shorter side) at 46.5 FPS / 34.0 AP; accuracy-priority (3 conv layers, 512px shorter side, deformable conv in the backbone’s last layer) at 31.3 FPS / 37.1 AP — both trained with the 3× schedule and scale jitter from $[352,512]$.
• Matrix NMS: applied post-hoc on masks already thresholded at sigmoid 0.5; confidence pre-filter at 0.1 before ranking.

Results

Mask AP on COCO test-dev, single model:

Method	Backbone	AP	AP${}_{50}$	AP${}_{75}$	AP${}_S$	AP${}_M$	AP${}_L$
Mask R-CNN	ResNet-101-FPN	35.7	58.0	37.8	15.5	38.1	52.4
TensorMask	ResNet-101-FPN	37.1	59.3	39.4	17.4	39.1	51.6
YOLACT	ResNet-101-FPN	31.2	50.6	32.8	12.1	33.3	47.1
BlendMask	ResNet-101-FPN	38.4	60.7	41.3	18.2	41.5	53.3
SOLO	ResNet-101-FPN	37.8	59.5	40.4	16.4	40.6	54.2
SOLOv2	ResNet-50-FPN	38.8	59.9	41.7	16.5	41.7	56.2
SOLOv2	ResNet-101-FPN	39.7	60.7	42.9	17.3	42.9	57.4
SOLOv2	ResNet-DCN-101-FPN	41.7	63.2	45.1	18.0	45.0	61.6

The gap over Mask R-CNN is largest on large objects (AP${}_L$: 56.2 vs. 52.4 for the ResNet-50 SOLOv2 vs. ResNet-101 Mask R-CNN) — direct evidence that Mask R-CNN’s RoI-relative, resolution-capped mask head loses detail on big objects that SOLOv2’s unified high-resolution mask-feature representation does not.

Fig. 1b (paper): Mask R-CNN’s mask head is capped at $28\times 28$ resolution before its output is resized back up, visibly blurring fine boundary detail (feather edges, fingers) that SOLOv2’s mask-feature-branch output preserves.

SOLOv2 also extends to object detection (44.9 box AP, computed by taking the bounding box of the predicted mask — no box-specific training at all) and panoptic segmentation (state of the art among box-free methods at the time, 42.1 PQ on COCO val2017 with a ResNet-50-FPN backbone), both without architectural changes to the core dynamic-segmentation mechanism.

Limitations

• Grid resolution is a hard assignment. Like SOLO, an object is only ever assigned to the grid cell containing its center; two same-class instances whose centers collide in the same cell at the same FPN level are unrecoverable ambiguity by construction, though multi-scale FPN assignment (larger objects to coarser levels) makes this rare in practice.
• Confidence-threshold pre-filtering (0.1) before Matrix NMS means the parallel formulation still depends on a reasonably well-calibrated per-cell classification score; Matrix NMS suppresses duplicates among survivors, but it does not fix a poorly calibrated category head.
• Unlike YOLACT’s prototype dictionary (shared, fixed-size, reused identically for every image), SOLOv2’s mask feature map $F$ is still recomputed per image, and the dynamic kernel — while cheap to predict — still requires one convolution per surviving instance; the saving over Mask R-CNN is in not needing per-instance RoI feature re-extraction, not in avoiding per-instance computation altogether.

See also

• Mask R-CNN — the RoI-based two-stage baseline SOLOv2 explicitly compares its mask detail against (Fig. 1b) and outperforms on large-object AP
• YOLACT — the other prominent box-free-adjacent real-time head; both decouple mask generation into two components (SOLOv2: kernel + feature map; YOLACT: coefficients + prototypes) combined by a cheap operation, but YOLACT still crops to a predicted box while SOLOv2 has no box dependency at all
• Segmentation Head Comparison — this head placed side by side with Mask R-CNN’s, YOLACT’s, DETR’s and Mask2Former’s
• DETR — a contemporaneous move away from anchors/RPNs on the detection side, arriving at a different (set-prediction) solution to the same “stop depending on box proposals” motivation