🎚️ Multitrack Support in Liquidsoap

Starting in version 2.2.0, Liquidsoap gained support for multitrack operations — a powerful addition that lets you work with individual tracks inside your media files. Whether it’s audio, video, or metadata, you can now demux, remux, encode, decode, apply filters, and more — all at the track level!

This unlocks advanced media workflows, like keeping multiple language tracks, adding a default video stream to audio-only files, or even mixing and matching content across sources.

Let’s walk through what this means, step by step 👇

🧠 What is a Track?

A media file can contain multiple “tracks.” Think of a movie file with:

• 🎧 English and French audio tracks
• 🎥 One video track
• 📝 Metadata like title and artist

In Liquidsoap, these tracks are made accessible through operators that let you manipulate them individually. This opens up many possibilities — but also introduces a few new concepts and conventions to learn.

🔧 Requirements: When You Need FFmpeg

Some multitrack features rely on FFmpeg, while others don’t:

To fully unlock multitrack functionality, make sure your Liquidsoap is compiled with FFmpeg support.

🎬 Using Multitrack Media

Let’s say you have a media file with multiple tracks, like:

movie.mkv
├── audio (English)
├── audio_2 (French)
└── video

You can create a source with this file like so:

s = single("/path/to/movie.mkv")

By default, only the first audio and video track will be used:

output.file(%ffmpeg(%audio.copy, %video.copy), "/path/to/copy.mkv", s)

🪵 Logs will confirm the detected tracks:

[output_file:3] Content type is {audio=ffmpeg.copy,video=ffmpeg.copy}.
[decoder.ffmpeg:3] Requested content-type for "/path/to/movie.mkv": {audio=ffmpeg.copy,video=ffmpeg.copy}
[decoder.ffmpeg:3] FFmpeg recognizes "/path/to/movie.mkv" as video: {codec: h264, 1920x1038, yuv420p}, audio: {codec: aac, 48000Hz, 6 channel(s)}, audio_2: {codec: aac, 48000Hz, 6 channel(s)}
[decoder.ffmpeg:3] Decoded content-type for "/path/to/movie.mkv": {audio=ffmpeg.copy(codec="aac",channel_layout="5.1",sample_format=fltp,sample_rate=48000),video=ffmpeg.copy(codec="h264",width=1920,height=1038,aspect_ratio=1/1,pixel_format=yuv420p)}

Here, audio_2 is present but unused. Let’s fix that 👇

🎛️ Custom Track Handling

What if you want to keep both audio tracks, re-encoding the second one to stereo?

output.file(
  %ffmpeg(
    %audio.copy,
    %audio_2(channels=2, codec="aac"),
    %video.copy
  ),
  "/path/to/copy.mkv",
  s
)

🪵 Logs now show audio_2 being processed:

[output_file:3] Content type is {audio=ffmpeg.copy,audio_2=pcm(stereo),video=ffmpeg.copy}.
[decoder.ffmpeg:3] Requested content-type for "/path/to/movie.mkv": {audio=ffmpeg.copy,audio_2=pcm(stereo),video=ffmpeg.copy}
[decoder.ffmpeg:3] FFmpeg recognizes "/path/to/movie.mkv" as video: {codec: h264, 1920x1038, yuv420p}, audio: {codec: aac, 48000Hz, 6 channel(s)}, audio_2: {codec: aac, 48000Hz, 6 channel(s)}
[decoder.ffmpeg:3] Decoded content-type for "/path/to/movie.mkv": {audio=ffmpeg.copy(codec="aac",channel_layout="5.1",sample_format=fltp,sample_rate=48000),audio_2=pcm(5.1),video=ffmpeg.copy(codec="h264",width=1920,height=1038,aspect_ratio=1/1,pixel_format=yuv420p)}

You now have a file with two audio tracks (one copied, one re-encoded) and one video track 🎉

⚠️ Playlist Caveats

If your source is a playlist:

s = playlist("/path/to/playlist")

…and you request multiple tracks:

output.file(
  fallible=true,
  %ffmpeg(%audio.copy, %audio_2(...), %video.copy),
  "/path/to/copy.mkv",
  s
)

Then only files with all requested tracks (audio + audio_2 + video) will be accepted. Others will be skipped.

🧭 Track Naming Conventions

When decoding, track names follow this pattern:

• audio, audio_2, audio_3, …
• video, video_2, video_3, …

These names are fixed by the decoder — you can’t rename them directly when reading media.

For example, this will fail:

output.file(%ffmpeg(%audio_fr.copy, %audio_en(...), "/path/to/file.mkv", playlist("...")))

Why? Because the decoder doesn’t know what audio_fr or audio_en means. Track names must match what the decoder emits: audio, audio_2, etc.

Once you’re remuxing tracks, however, you can assign any name you want!

🔄 Demuxing and Remuxing Tracks

To extract and rebuild track sets:

s = playlist(...)

let {audio, video, metadata, track_marks} = source.tracks(s)

You can then remix these into a new source:

s = source({
  audio = audio,
  video = video,
  metadata = metadata,
  track_marks = track_marks
})

Tracks can also be added or replaced:

s = source(source.tracks(s).{video = source.tracks(image).video})

One limitation is that it is not currently possible to add default tracks.

The following won’t work:

video = source.tracks(s).video ?? source.tracks(image).video

🧹 Cleaning Up Tracks

Want to remove track_marks?

let {track_marks=_, ...tracks} = source.tracks(s)
s = source(tracks)

This is equivalent to the older drop_tracks operator.

🔌 Track-Level Operators

Many operators now work directly on tracks. Examples:

mono = track.audio.mean(audio_track)
encoded = track.ffmpeg.encode.audio(%ffmpeg(%audio(codec="aac")), audio_track)

🚨 But beware: some operators (like inline encoders) put the track on a new clock. You’ll need to re-derive metadata and track marks from the new track to avoid clock conflicts:

let encoded_audio = track.ffmpeg.encode.audio(..., audio)

s = source({
  audio = encoded_audio,
  metadata = track.metadata(encoded_audio),
  track_marks = track.track_marks(encoded_audio)
})

🏷️ How Encoders Detect Track Types

Liquidsoap uses naming conventions and hints to determine what kind of data each track holds:

Priority order:

1. %audio.copy or %video.copy → auto-detect
2. Named content type (audio_content or video_content)
3. Track name contains “audio” or “video”
4. Codec implies type

✅ Example: Explicit typing

output.file(
  %ffmpeg(
    %en(audio_content, codec=audio_codec),
    %fr(codec="aac"),
    %director_cut(video_content, codec=video_codec)
  ),
  "/path/to/copy.mkv",
  s
)

✅ Or by naming convention

%audio_en(codec=...)
%director_cut_video(codec=...)

Internally, Liquidsoap maps this to content-type info. Once handed off to FFmpeg, track names are lost — FFmpeg just sees numbered tracks in order.

🚀 Summary

Multitrack support opens up powerful new workflows:

• 🔄 Mix and match audio/video/metadata across sources
• 🧱 Build custom media containers
• 🎯 Target different formats per track
• 🧪 Combine synchronous and asynchronous operations (with care!)

We’ve only scratched the surface — go ahead and explore the code, experiment, and let your creativity flow! 💡

And if there’s an operator you wish worked at the track level, don’t hesitate to open a feature request!