Input: A regular color photo (PNG, JPEG, etc.)
<aside> ðŸ›
Find Objects and Cut Them Out, Pixel by Pixel
The "cutout" version of RF-DETR-Medium. Same idea - finds objects in a photo - but instead of just drawing a box around each one, it traces the exact outline of every object, pixel by pixel. Great for cutouts, background removal, compositing, or anything that needs precise object shapes rather than just rectangles
</aside>
RF-DETR-Seg-Medium looks at a photo and, for every object it spots, returns four pieces of information: a label ("person", "car", "dog"), a box around the object, a confidence score, and - the new part - a pixel-precise mask showing the exact shape of the object. Where the regular detector says "there's a dog inside this rectangle," this one says "here are the exact pixels that belong to the dog." It even handles overlapping objects of the same kind - two cats on a couch get two separate masks. It recognizes the same 80 everyday categories as the detection sibling (people, animals, vehicles, household items, food, etc.). The "Medium" size is the balanced choice between accuracy and speed.
<aside>
Input: A regular color photo (PNG, JPEG, etc.)
</aside>
<aside>
<aside>
Output: For every object the model finds:
| Model size | 35.7 million parameters — slightly bigger than the detection sibling (which doesn't need to draw masks) |
| Speed | Real-time on modern GPUs; the "Medium" pick on the speed-vs-accuracy spectrum |
| Where it sits in the family | RF-DETR-Seg comes in multiple sizes — Medium is the balanced default |
| How it compares to YOLO | At the same speed bracket, RF-DETR-Seg-Medium produces more accurate masks than similarly fast YOLO-seg models (e.g., +1.2 mask AP over YOLO26m-seg at slightly faster speed). The trade-off: RF-DETR uses more parameters, so it needs more memory than YOLO of the same "size letter" |
| What it recognizes | 80 everyday object categories (the standard COCO list) |
| What you get per object | Label + confidence + box + pixel-precise mask, all in a single pass |
| Good for | Production segmentation where mask quality matters and you want better results than YOLO-seg at similar speed |
| Architecture | RF-DETR (segmentation variant) — DINOv2-with-registers ViT backbone + multi-scale projector + Deformable DETR decoder + instance-segmentation head |
| Backbone Detail | DINOv2-with-registers style ViT with RF-DETR's windowed / full attention alternation |
| Multi-scale Fusion | RF-DETR multi-scale projector (C2f-style blocks, LW-DETR lineage) aggregates multi-level backbone features before the decoder |
| Decoder | Deformable DETR-style decoder with multi-scale deformable cross-attention; mask prediction head added on top of box/class outputs |
| Object Queries | DETR-style queries with bipartite matching and auxiliary decoder losses |
| Training Losses | Box (L1 + GIoU), classification, and mask losses (focal / dice-style terms as configured), with auxiliary supervision on intermediate decoder layers |
| Training Tricks | Group DETR (parallel decoder copies for faster convergence) |
| Family Discovery | Weight-sharing Neural Architecture Search (NAS) over accuracy/latency knobs — Seg-Medium is one subnet on the frontier |
| Parameters | 35.7M (F32 weights) |
| Training Data | COCO 2017 instance segmentation (80 thing categories, as reflected in config.id2label) |
| Supported Frameworks | Hugging Face Transformers (RfDetrForInstanceSegmentation), Safetensors |
| Paper | "RF-DETR: Neural Architecture Search for Real-Time Detection Transformers" — arXiv:2511.09554 (Robinson et al., Nov 2025) |
| Sibling model | RF-DETR-Medium (detection only) — same architecture without the |