Multimodal Language Models - Molmo2-8B

<aside> 🛠

Open Vision-Language Model

Molmo2-8B understands a text prompt plus up to two images in a single pass, and is state-of-the-art among open multimodal models in its 8B size class on counting, captioning, and visual question answering.

</aside>

What it does

Molmo2-8B is a multimodal AI from the Allen Institute for AI (Ai2) that reads images and answers questions or generates captions in natural language. Given a text prompt with one or two reference images, it can describe what it sees, answer questions about the content, compare two images, count objects, or extract structured information. The model is built on Qwen3-8B as the language backbone with SigLIP 2 as the vision encoder, trained on the publicly released Molmo2 data collection of highly curated image-text pairs. The CNAPS Studio integration runs the bf16 variant in ~17GB VRAM.

Problem it solves

Visual question answering (VQA) – Answer natural-language questions about image content with strong accuracy across everyday scenes, documents, charts, and product photos
Image captioning at scale – Generate descriptive, accurate captions for single images — useful for accessibility, content indexing, and dataset enrichment
Two-image comparison and analysis – Pass a primary image plus an optional second image to compare, contrast, or describe their relationship in one model call (e.g., before/after shots, A/B product comps)
Counting and enumeration – Best-in-class among open 8B-scale multimodal models at counting objects and listing distinct items
Cost-efficient open-weights deployment – At ~17GB VRAM in bf16, runs on a single mid-range GPU and is fully Apache 2.0 — no API fees, no rate limits, no data leaving your infrastructure

Input/Output

<aside>

Input:
- Required input:
  - Text prompt – Question, instruction, or caption request (e.g., "Describe this image", "What is the person doing?", "Compare these two images")
- Optional inputs:
  - Image 1 – Primary reference image
  - Image 2 – Second reference image for comparison or multi-image questions
- Note: All inputs accept standard image formats (PNG, JPEG). Prompts work with one or both images, or text-only if you want to query the model conversationally. </aside>

<aside>

Advanced configuration:

Parameter	Range	Default	What it does
Max new tokens	1 ~ 4096	512	Maximum length of the model's response. Increase for long captions or detailed analyses; lower it for short answers and faster generation
Temperature	0 ~ 2	0.7	Controls randomness. Lower (0.2–0.5) for factual VQA and consistent captioning; higher (0.8–1.2) for varied, creative descriptions
Top-p (nucleus)	0 ~ 1	0.95	Restricts sampling to tokens whose cumulative probability adds up to this threshold. Lower = more focused; default 0.95 is a good general setting
Top-k	integer	50	Limits sampling to the top-K most likely tokens at each step. Works alongside Top-p; 50 is a balanced default
</aside>

<aside>

Output:
- Text – Natural-language answer, caption, comparison, or structured response based on the prompt

</aside>

Accuracy & Speed

Benchmark Performance

Average Score on 15 Academic Video Benchmarks (from Ai2's technical report)

Model	Score
Molmo2-8B (this model)	63.1
Molmo2-4B	62.8
Eagle2.5-8B	60.7
Qwen3-VL-8B	59.5
Molmo2-7B	59.7
GLM-4.1V-9B	56.9
MiniCPM-V-4.5-8B	56.6
InternVL3.5-8B	54.1

Reference closed-source models (for context, not direct competitors in the open-weights class)

Model	Score
Gemini 2.5 Pro	71.2
GPT-5	70.6
Gemini 3 Pro	70.0
GPT-5 mini	65.0
Claude Sonnet 4.5	59.6