Audio Understanding - Speech Recognition (Clips)

<aside> 🛠

Recognize Speech in Each Clip of a Video List and Convert Them to Timed Transcripts

Split a video into scenes or highlights and you’re left with a pile of clips that still need captioning one by one. This node transcribes the whole list in a single pass — feed it the clips from an upstream splitter and get back a timed transcript for each one, timestamps preserved so captions line up exactly. It runs on OpenAI’s Whisper Large V3, the same state-of-the-art open-source recognizer used by the Video variant. Language auto-detects by default, or force English, Korean, Japanese, or Chines.

🏆 7.44 mean WER on the Open ASR Leaderboard, running at 145.51× real-time (RTFx)

</aside>

What it does

This node takes a list of video clips (not a single video — a list of them, typically produced upstream by a scene-detection, highlight extraction, or clip-splitting node) and, for each clip, extracts the audio and produces a timed transcript. The output is per-clip, with timestamps preserved — so you know exactly when each phrase was spoken within each clip. Under the hood the model is the same OpenAI Whisper Large V3 used by the Video variant — the same 99-language coverage, the same accuracy, the same robustness to accents and noise. What makes this node different is the list-in, per-clip-timed-transcript-out shape, which is designed for downstream operations that need to line up captions, subtitles, or search markers against a set of segments. If you just have one video and want one transcript, use Audio Understanding - Speech Recognition (Video) instead.

Problem it solves

Batch clip transcription – Transcribe every clip in a list in one pass, without wiring separate speech recognition nodes per clip
Timed subtitle / caption generation – Get per-clip timestamps for driving subtitle tracks or on-screen captions in downstream video editing
Per-scene / per-highlight transcripts – When an upstream node has already split a video into scenes or highlights, this node fills each of them with a text transcript aligned by time
Searchable video segments – Make individual clips searchable by their transcript text, with timestamps
Multilingual clip lists – Auto language detection works per clip, so mixed-language clip lists are handled naturally
Robust to real-world audio – Same Whisper Large V3 strengths — background noise, accents, technical vocabulary

Input/Output

<aside>

Input: A list of video clips (as produced by an upstream splitter / scene detector / highlight extractor node) </aside>

<aside>

Output: A list of timed transcripts, one per input clip
- Each transcript is timestamped so downstream nodes can align the text with the original video timeline (sentence-level or word-level timestamps depending on downstream needs) </aside>

<aside>

Options:
- Spoken language (dropdown, default Auto)
  - Auto — model detects the language from the audio itself (recommended for mixed or unknown-language content)
  - English — force English
  - Korean — force Korean
  - Japanese — force Japanese
  - Chinese — force Chinese
  - Note: under the hood the model supports 99 languages; the studio's picker is scoped to the four most-used explicit choices
- Output mode (dropdown, default Transcribe)
  - Transcribe — keeps the spoken language (audio in language X → text in language X)
  - Translate to English — uses Whisper's native speech-to-English mode (audio in any language → English text)
  - Tip: run the same video twice (Transcribe + Translate to English) to get both transcripts for a bilingual subtitle stack. </aside>

Accuracy & Speed


Model	OpenAI Whisper Large V3 — state-of-the-art open-source speech recognition
Mean WER (Open ASR Leaderboard)	7.44 — among the strongest open ASR models
Real-Time Factor (RTFx)	145.51 — much faster than real-time on capable hardware
Improvement over Whisper V2	10 – 20% fewer errors across most languages
Languages	99 supported by the model; UI-exposed choices: Auto, English, Korean, Japanese, Chinese
Timing granularity	Sentence-level or word-level timestamps supported by the model
Model size	1.55B parameters
Throughput on clip lists	Whisper's chunked inference scales well across batched clips; typically much faster than real-time in aggregate

How the Clips variant compares

	Speech Recognition (Clips)	Speech Recognition (Video)
Input	A list of video clips	A single video file
Output	A timed transcript per clip	One plain-text transcript
Best for	Per-scene / per-highlight captioning, subtitle alignment	One video → one transcript
Underlying model	Whisper Large V3	Whisper Large V3

Both run inside CNAPS Studio — for the single-video workflow use the sibling Audio Understanding - Speech Recognition (Video) node, or compare them side-by-side.

Technical Details (for engineers)