Loading source
Pulling the file list, source metadata, and syntax-aware rendering for this listing.
Source from repo
Apply best practices for creating programmatic videos with Remotion and React.
Files
Skill
Size
Entrypoint
Format
Open file
Syntax-highlighted preview of this file as included in the skill package.
rules/display-captions.md
1---2name: display-captions3description: Displaying captions in Remotion with TikTok-style pages and word highlighting4metadata:5tags: captions, subtitles, display, tiktok, highlight6---78# Displaying captions in Remotion910This guide explains how to display captions in Remotion, assuming you already have captions in the [`Caption`](https://www.remotion.dev/docs/captions/caption) format.1112## Prerequisites1314Read [Transcribing audio](transcribe-captions.md) for how to generate captions.1516First, the [`@remotion/captions`](https://www.remotion.dev/docs/captions) package needs to be installed.17If it is not installed, use the following command:1819```bash20npx remotion add @remotion/captions21```2223## Fetching captions2425First, fetch your captions JSON file. Use [`useDelayRender()`](https://www.remotion.dev/docs/use-delay-render) to hold the render until the captions are loaded:2627```tsx28import { useState, useEffect, useCallback } from "react";29import { AbsoluteFill, staticFile, useDelayRender } from "remotion";30import type { Caption } from "@remotion/captions";3132export const MyComponent: React.FC = () => {33const [captions, setCaptions] = useState<Caption[] | null>(null);34const { delayRender, continueRender, cancelRender } = useDelayRender();35const [handle] = useState(() => delayRender());3637const fetchCaptions = useCallback(async () => {38try {39// Assuming captions.json is in the public/ folder.40const response = await fetch(staticFile("captions123.json"));41const data = await response.json();42setCaptions(data);43continueRender(handle);44} catch (e) {45cancelRender(e);46}47}, [continueRender, cancelRender, handle]);4849useEffect(() => {50fetchCaptions();51}, [fetchCaptions]);5253if (!captions) {54return null;55}5657return <AbsoluteFill>{/* Render captions here */}</AbsoluteFill>;58};59```6061## Creating pages6263Use `createTikTokStyleCaptions()` to group captions into pages. The `combineTokensWithinMilliseconds` option controls how many words appear at once:6465```tsx66import { useMemo } from "react";67import { createTikTokStyleCaptions } from "@remotion/captions";68import type { Caption } from "@remotion/captions";6970// How often captions should switch (in milliseconds)71// Higher values = more words per page72// Lower values = fewer words (more word-by-word)73const SWITCH_CAPTIONS_EVERY_MS = 1200;7475const { pages } = useMemo(() => {76return createTikTokStyleCaptions({77captions,78combineTokensWithinMilliseconds: SWITCH_CAPTIONS_EVERY_MS,79});80}, [captions]);81```8283## Rendering with Sequences8485Map over the pages and render each one in a `<Sequence>`. Calculate the start frame and duration from the page timing:8687```tsx88import { Sequence, useVideoConfig, AbsoluteFill } from "remotion";89import type { TikTokPage } from "@remotion/captions";9091const CaptionedContent: React.FC = () => {92const { fps } = useVideoConfig();9394return (95<AbsoluteFill>96{pages.map((page, index) => {97const nextPage = pages[index + 1] ?? null;98const startFrame = (page.startMs / 1000) * fps;99const endFrame = Math.min(100nextPage ? (nextPage.startMs / 1000) * fps : Infinity,101startFrame + (SWITCH_CAPTIONS_EVERY_MS / 1000) * fps,102);103const durationInFrames = endFrame - startFrame;104105if (durationInFrames <= 0) {106return null;107}108109return (110<Sequence111key={index}112from={startFrame}113durationInFrames={durationInFrames}114>115<CaptionPage page={page} />116</Sequence>117);118})}119</AbsoluteFill>120);121};122```123124## White-space preservation125126The captions are whitespace sensitive. You should include spaces in the `text` field before each word. Use `whiteSpace: "pre"` to preserve the whitespace in the captions.127128## Separate component for captions129130Put captioning logic in a separate component.131Make a new file for it.132133## Word highlighting134135A caption page contains `tokens` which you can use to highlight the currently spoken word:136137```tsx138import { AbsoluteFill, useCurrentFrame, useVideoConfig } from "remotion";139import type { TikTokPage } from "@remotion/captions";140141const HIGHLIGHT_COLOR = "#39E508";142143const CaptionPage: React.FC<{ page: TikTokPage }> = ({ page }) => {144const frame = useCurrentFrame();145const { fps } = useVideoConfig();146147// Current time relative to the start of the sequence148const currentTimeMs = (frame / fps) * 1000;149// Convert to absolute time by adding the page start150const absoluteTimeMs = page.startMs + currentTimeMs;151152return (153<AbsoluteFill style={{ justifyContent: "center", alignItems: "center" }}>154<div style={{ fontSize: 80, fontWeight: "bold", whiteSpace: "pre" }}>155{page.tokens.map((token) => {156const isActive =157token.fromMs <= absoluteTimeMs && token.toMs > absoluteTimeMs;158159return (160<span161key={token.fromMs}162style={{ color: isActive ? HIGHLIGHT_COLOR : "white" }}163>164{token.text}165</span>166);167})}168</div>169</AbsoluteFill>170);171};172```173174## Display captions alongside video content175176By default, put the captions alongside the video content, so the captions are in sync.177For each video, make a new captions JSON file.178179```tsx180<AbsoluteFill>181<Video src={staticFile("video.mp4")} />182<CaptionPage page={page} />183</AbsoluteFill>184```185