Extended subtitles support across containers

docs/guide/converting-media-files.md

@@ -294,11 +294,46 @@ The function is called for each input audio sample after remixing and resampling
 
 This function can also be used to manually perform remixing or resampling. When doing so, you should signal the post-process parameters using the `processedNumberOfChannels` and `processedSampleRate` fields, which enables the encoder to better know what to expect.
 
+## Subtitle options
+
+You can set the `subtitle` property in the conversion options to configure the converter's behavior for subtitle tracks. The options are:
+```ts
+type ConversionSubtitleOptions = {
+	discard?: boolean;
+	codec?: SubtitleCodec;
+};
+```
+
+For example, here we convert all subtitle tracks to WebVTT format:
+```ts
+const conversion = await Conversion.init({
+	input,
+	output,
+	subtitle: {
+		codec: 'webvtt',
+	},
+});
+```
+
+::: info
+The provided configuration will apply equally to all subtitle tracks of the input. If you want to apply a separate configuration to each subtitle track, check [track-specific options](#track-specific-options).
+:::
+
+### Discarding subtitles
+
+If you want to get rid of subtitle tracks, use `discard: true`.
+
+### Converting subtitle format
+
+Use the `codec` property to control the format of the output subtitle tracks. This should be set to a [codec](./supported-formats-and-codecs#subtitle-codecs) supported by the output file, or else the track will be [discarded](#discarded-tracks).
+
+Subtitle tracks are always copied (extracted and re-muxed as text), never transcoded, so there is no quality loss. The supported formats are WebVTT, SRT, ASS/SSA, TX3G, and TTML.
+
 ## Track-specific options
 
-You may want to configure your video and audio options differently depending on the specifics of the input track. Or, in case a media file has multiple video or audio tracks, you may want to discard only specific tracks or configure each track separately.
+You may want to configure your video, audio, and subtitle options differently depending on the specifics of the input track. Or, in case a media file has multiple tracks of the same type, you may want to discard only specific tracks or configure each track separately.
 
-For this, instead of passing an object for `video` and `audio`, you can instead pass a function:
+For this, instead of passing an object for `video`, `audio`, or `subtitle`, you can instead pass a function:
 
 ```ts
 const conversion = await Conversion.init({
@@ -329,10 +364,22 @@ const conversion = await Conversion.init({
 			codec: 'aac',
 		};
 	},
+
+	// Works for subtitles too:
+	subtitle: (subtitleTrack, n) => {
+		if (subtitleTrack.languageCode !== 'eng' && subtitleTrack.languageCode !== 'spa') {
+			// Keep only English and Spanish subtitles
+			return { discard: true };
+		}
+
+		return {
+			codec: 'webvtt',
+		};
+	},
 });
 ```
 
-For documentation about the properties of video and audio tracks, refer to [Reading track metadata](./reading-media-files#reading-track-metadata).
+For documentation about the properties of video, audio, and subtitle tracks, refer to [Reading track metadata](./reading-media-files#reading-track-metadata).
 
 ## Trimming
 

docs/guide/supported-formats-and-codecs.md

@@ -64,6 +64,11 @@ Mediabunny ships with built-in decoders and encoders for all audio PCM codecs, m
 ### Subtitle codecs
 
 - `'webvtt'` - WebVTT
+- `'tx3g'` - 3GPP Timed Text (MP4 subtitles)
+- `'ttml'` - Timed Text Markup Language
+- `'srt'` - SubRip
+- `'ass'` - Advanced SubStation Alpha
+- `'ssa'` - SubStation Alpha
 
 ## Compatibility table
 
@@ -97,11 +102,15 @@ Not all codecs can be used with all containers. The following table specifies th
 | `'pcm-f64be'`  |    ✓     |   ✓   |       |           |       |       |       |       |       |       |
 | `'ulaw'`       |          |   ✓   |       |           |       |       |   ✓   |       |       |       |
 | `'alaw'`       |          |   ✓   |       |           |       |       |   ✓   |       |       |       |
-| `'webvtt'`[^3] |   (✓)    |       |  (✓)  |    (✓)    |       |       |       |       |       |       |
+| `'webvtt'`     |    ✓     |   ✓   |   ✓   |     ✓     |       |       |       |       |       |       |
+| `'tx3g'`       |    ✓     |   ✓   |       |           |       |       |       |       |       |       |
+| `'ttml'`       |    ✓     |   ✓   |       |           |       |       |       |       |       |       |
+| `'srt'`        |          |       |   ✓   |     ✓     |       |       |       |       |       |       |
+| `'ass'`        |          |       |   ✓   |     ✓     |       |       |       |       |       |       |
+| `'ssa'`        |          |       |   ✓   |     ✓     |       |       |       |       |       |       |
 
 
 [^2]: WebM only supports a small subset of the codecs supported by Matroska. However, this library can technically read all codecs from a WebM that are supported by Matroska.
-[^3]: WebVTT can only be written, not read.
 
 ## Querying codec encodability
 
@@ -330,4 +339,4 @@ All instance methods of the class can return promises. In this case, the library
 
 ::: warning
 The samples passed to `onSample` **must** be sorted by increasing timestamp. This especially means if the decoder is decoding a video stream that makes use of [B-frames](./media-sources.md#b-frames), the decoder **must** internally hold on to these frames so it can emit them sorted by presentation timestamp. This strict sorting requirement is reset each time `flush` is called.
-:::
+:::

examples/subtitle-extraction/index.html

@@ -0,0 +1,94 @@
+<!DOCTYPE html>
+<html lang="en" translate="no">
+	<head>
+		<meta charset="UTF-8">
+		<meta http-equiv="X-UA-Compatible" content="IE=edge">
+		<meta name="viewport" content="width=device-width, initial-scale=1.0">
+		<title>Subtitle extraction example | Mediabunny</title>
+		<script type="module" src="../base.ts"></script>
+		<script type="module" src="./subtitle-extraction.ts"></script>
+		<link rel="stylesheet" href="../base.css">
+		<link rel="icon" href="../../docs/public/mediabunny-logo.svg">
+	</head>
+
+	<body class="flex flex-col items-center py-10 bg-zinc-50 text-zinc-800 dark:bg-zinc-900 dark:text-zinc-200 px-2">
+		<h1 class="text-3xl font-bold text-blue-500 text-center">Subtitle extraction example</h1>
+		<p class="max-w-lg text-center">Extract subtitle tracks from video files with embedded subtitles (MKV, MP4, MOV).</p>
+
+		<div class="flex flex-col items-center gap-1">
+			<div class="flex gap-2 mt-4">
+				<button id="select-file" class="rounded-lg bg-zinc-200 dark:bg-zinc-750 hover:bg-zinc-300 dark:hover:bg-zinc-700 px-5 py-2">
+					Select local file
+				</button>
+
+				<button id="load-url" class="rounded-lg bg-zinc-200 dark:bg-zinc-750 hover:bg-zinc-300 dark:hover:bg-zinc-700 px-5 py-2">
+					Load remote URL
+				</button>
+			</div>
+
+			<a id="sample-file-download" download="sample.mkv" class="hover:underline text-xs opacity-50 hover:opacity-70">
+				Download sample file
+			</a>
+		</div>
+
+		<p class="text-xs opacity-60 mt-2" id="file-name"></p>
+
+		<hr class="w-full max-w-96 my-4 border-zinc-300 dark:border-zinc-700" style="display: none;">
+
+		<div id="content-container" class="flex flex-col items-center gap-4 w-full max-w-4xl"></div>
+
+		<a href="/" class="fixed top-0 left-0 flex gap-2 py-2 px-5 items-center">
+			<img src="../../docs/public/mediabunny-logo.svg" class="size-6">
+			<p class="text-sm font-medium">Mediabunny</p>
+		</a>
+
+		<a
+			href="https://github.com/Vanilagy/mediabunny/tree/main/examples/subtitle-extraction"
+			target="_blank"
+			class="flex items-center gap-2 fixed top-0 right-0 py-2 px-5 bg-zinc-200 dark:bg-zinc-750 hover:bg-zinc-300 dark:hover:bg-zinc-700 rounded-bl-xl"
+		>
+			<img src="../../docs/assets/github-mark.svg" class="size-6 dark:invert">
+			<p>View source code</p>
+		</a>
+	</body>
+
+	<style>
+		@reference "../base.css";
+
+		.subtitle-track {
+			@apply py-4 px-6 bg-zinc-100 dark:bg-zinc-800 rounded-lg w-full;
+		}
+
+		.subtitle-track-header {
+			@apply flex justify-between items-center mb-3;
+		}
+
+		.subtitle-track-title {
+			@apply text-lg font-semibold;
+		}
+
+		.subtitle-track-meta {
+			@apply text-sm opacity-60;
+		}
+
+		.cue-preview {
+			@apply mt-2 p-3 bg-white dark:bg-zinc-900 rounded border border-zinc-200 dark:border-zinc-700 max-h-64 overflow-y-auto;
+		}
+
+		.cue-item {
+			@apply border-b border-zinc-200 dark:border-zinc-700 py-2 text-sm;
+		}
+
+		.cue-item:last-child {
+			@apply border-b-0;
+		}
+
+		.cue-time {
+			@apply text-blue-500 font-mono text-xs mr-2;
+		}
+
+		.download-btn {
+			@apply mt-3 px-4 py-2 bg-blue-500 text-white rounded-lg hover:bg-blue-600 text-sm;
+		}
+	</style>
+</html>

examples/subtitle-extraction/subtitle-extraction.ts

@@ -0,0 +1,184 @@
+import { Input, ALL_FORMATS, BlobSource, UrlSource } from 'mediabunny';
+
+// Sample file URL - users can replace with their own
+const SampleFileUrl = 'https://commondatastorage.googleapis.com/gtv-videos-bucket/sample/BigBuckBunny.mp4';
+(document.querySelector('#sample-file-download') as HTMLAnchorElement).href = SampleFileUrl;
+
+const selectMediaButton = document.querySelector('#select-file') as HTMLButtonElement;
+const loadUrlButton = document.querySelector('#load-url') as HTMLButtonElement;
+const fileNameElement = document.querySelector('#file-name') as HTMLParagraphElement;
+const horizontalRule = document.querySelector('hr') as HTMLHRElement;
+const contentContainer = document.querySelector('#content-container') as HTMLDivElement;
+
+const extractSubtitles = async (resource: File | string) => {
+	fileNameElement.textContent = resource instanceof File ? resource.name : resource;
+	horizontalRule.style.display = '';
+	contentContainer.innerHTML = '<p class="text-sm opacity-60">Loading...</p>';
+
+	try {
+		const source = resource instanceof File
+			? new BlobSource(resource)
+			: new UrlSource(resource);
+
+		const input = new Input({
+			source,
+			formats: ALL_FORMATS,
+		});
+
+		const subtitleTracks = await input.subtitleTracks;
+
+		if (!subtitleTracks || subtitleTracks.length === 0) {
+			contentContainer.innerHTML = '<p class="text-sm opacity-60">No subtitle tracks found in this file.</p>';
+			input.dispose();
+			return;
+		}
+
+		// Extract all subtitle data before disposing input
+		const subtitleData = await Promise.all(subtitleTracks.map(async (track) => {
+			const cues = [];
+			let cueCount = 0;
+			for await (const cue of track.getCues()) {
+				cues.push(cue);
+				cueCount++;
+				if (cueCount >= 5) break;
+			}
+
+			// Get full text for download
+			const fullText = await track.exportToText();
+
+			return {
+				id: track.id,
+				name: track.name,
+				codec: track.codec,
+				languageCode: track.languageCode,
+				previewCues: cues,
+				fullText,
+			};
+		}));
+
+		// Now dispose the input
+		input.dispose();
+
+		// Render subtitle tracks
+		contentContainer.innerHTML = '';
+
+		for (const trackData of subtitleData) {
+			const trackDiv = document.createElement('div');
+			trackDiv.className = 'subtitle-track';
+
+			// Header
+			const headerDiv = document.createElement('div');
+			headerDiv.className = 'subtitle-track-header';
+
+			const titleSpan = document.createElement('span');
+			titleSpan.className = 'subtitle-track-title';
+			titleSpan.textContent = trackData.name || `Track ${trackData.id}`;
+
+			const metaSpan = document.createElement('span');
+			metaSpan.className = 'subtitle-track-meta';
+			metaSpan.textContent = `${trackData.codec?.toUpperCase()} • ${trackData.languageCode}`;
+
+			headerDiv.appendChild(titleSpan);
+			headerDiv.appendChild(metaSpan);
+			trackDiv.appendChild(headerDiv);
+
+			// Cue preview
+			const previewDiv = document.createElement('div');
+			previewDiv.className = 'cue-preview';
+
+			if (trackData.previewCues.length > 0) {
+				for (const cue of trackData.previewCues) {
+					const cueDiv = document.createElement('div');
+					cueDiv.className = 'cue-item';
+
+					const timeSpan = document.createElement('span');
+					timeSpan.className = 'cue-time';
+					timeSpan.textContent = formatTime(cue.timestamp);
+
+					const textSpan = document.createElement('span');
+					textSpan.textContent = cue.text.substring(0, 100) + (cue.text.length > 100 ? '...' : '');
+
+					cueDiv.appendChild(timeSpan);
+					cueDiv.appendChild(textSpan);
+					previewDiv.appendChild(cueDiv);
+				}
+
+				const countNote = document.createElement('p');
+				countNote.className = 'text-xs opacity-50 mt-2';
+				countNote.textContent = `Showing first ${trackData.previewCues.length} cues`;
+				previewDiv.appendChild(countNote);
+			} else {
+				previewDiv.innerHTML = '<p class="text-xs opacity-50">No cues found</p>';
+			}
+
+			trackDiv.appendChild(previewDiv);
+
+			// Download button
+			const downloadBtn = document.createElement('button');
+			downloadBtn.className = 'download-btn';
+			downloadBtn.textContent = `Download as ${trackData.codec?.toUpperCase()}`;
+			downloadBtn.onclick = () => {
+				try {
+					const blob = new Blob([trackData.fullText], { type: 'text/plain' });
+					const url = URL.createObjectURL(blob);
+					const a = document.createElement('a');
+					a.href = url;
+					a.download = `subtitles_track${trackData.id}.${trackData.codec}`;
+					a.click();
+					URL.revokeObjectURL(url);
+				} catch (err) {
+					alert(`Error: ${err}`);
+				}
+			};
+			trackDiv.appendChild(downloadBtn);
+
+			contentContainer.appendChild(trackDiv);
+		}
+	} catch (err) {
+		console.error(err);
+		contentContainer.innerHTML = `<p class="text-red-500 text-sm">Error: ${err}</p>`;
+	}
+};
+
+const formatTime = (seconds: number): string => {
+	const h = Math.floor(seconds / 3600);
+	const m = Math.floor((seconds % 3600) / 60);
+	const s = Math.floor(seconds % 60);
+	const ms = Math.floor((seconds % 1) * 1000);
+	return `${h.toString().padStart(2, '0')}:${m.toString().padStart(2, '0')}:${s.toString().padStart(2, '0')}.${ms.toString().padStart(3, '0')}`;
+};
+
+selectMediaButton.addEventListener('click', () => {
+	const fileInput = document.createElement('input');
+	fileInput.type = 'file';
+	fileInput.accept = 'video/*,video/x-matroska,video/x-msvideo';
+	fileInput.addEventListener('change', () => {
+		const file = fileInput.files?.[0];
+		if (!file) return;
+		void extractSubtitles(file);
+	});
+	fileInput.click();
+});
+
+loadUrlButton.addEventListener('click', () => {
+	const url = prompt(
+		'Enter URL of a media file with subtitles. Must support CORS.',
+		'https://commondatastorage.googleapis.com/gtv-videos-bucket/sample/BigBuckBunny.mp4',
+	);
+	if (!url) return;
+	void extractSubtitles(url);
+});
+
+document.addEventListener('dragover', (event) => {
+	event.preventDefault();
+	event.dataTransfer!.dropEffect = 'copy';
+});
+
+document.addEventListener('drop', (event) => {
+	event.preventDefault();
+	const files = event.dataTransfer?.files;
+	const file = files && files.length > 0 ? files[0] : undefined;
+	if (file) {
+		void extractSubtitles(file);
+	}
+});