<aside> 🛠

Language-Prompted Image Segmentation

Segment every instance of an open-vocabulary concept in an image using a short text phrase like "person" or "red car". SAM 3.1 is the Object Multiplex refresh of SAM 3 - same prompting interface and concept coverage, with improved detection scores over SAM 3.

</aside>

What it does

SAM 3.1 (Scene) segments objects in images using text prompts (or visual prompts: points, boxes, masks). Point it at a photo, type a short phrase — "cat", "person wearing glasses", "yellow car" - and it automatically finds and masks all matching objects in the scene. Architecturally it's the next-generation drop-in upgrade to SAM 3: the prompting model and 4M+ concept vocabulary carry over, but the new Object Multiplex memory design improves detection scores on the SA-Co and VEval benchmarks over SAM 3 at comparable inference cost. Use this when you would have used SAM 3 - you'll get the same workflow with better accuracy.

Problem it solves

• Better detection accuracy than SAM 3 – Object Multiplex's shared-memory design improves SA-Co / VEval scores over SAM 3 without changing how you prompt the model
• Find specific objects with text – Describe what to find ("person", "cat in red", "car on the left") and segment all matching objects in one pass
• Understand attributes – Distinguishes "person in white shirt" from "person in red shirt"
• Understand spatial relationships – Interprets "left", "right", "background", "foreground"
• Privacy filtering – Automatically blur faces, license plates, screens, or documents
• Image annotation – Generate precise masks for every instance of a concept
• Object counting – Identify and count all instances of a given type
• Content moderation – Detect and mask sensitive content by description

Input/Output

<aside>

• Input: Images with text prompts (or visual prompts)
  • Format: PNG, JPEG, BMP, GIF
  • Content: Photos, screenshots, documents, surveillance footage, artwork
  • Prompts: Short text phrases ("person", "cat in red", "car on the left")
  • Visual prompts: Points, bounding boxes, or example masks </aside>

<aside>

• Options:
  • Merge to binary mask (checkbox) - when on, all per-instance masks are flattened into a single binary mask for the concept </aside>

<aside>

• Output: Segmentation masks with instance IDs
  • Masks: Binary or confidence masks for each matching object (or a single merged binary mask when "Merge to binary mask" is on)
  • Coordinates: Bounding boxes and exact object boundaries
  • Confidence scores: Model confidence per mask </aside>

Performance

Technical Details