FFmpeg Support

Since the 2.0.x release cycle, liquidsoap includes tight FFmpeg integration. This covers:

• Decoders
• Encoders
• Filters
• Subtitles
• Multitrack
• Bitstream filters
• Encoded data tweaks
• Examples

FFmpeg support includes 3 types of content:

• Internal content, that is content available to all liquidsoap operators: PCM audio and YUV420p video
• Raw content, that is decoded content stored as ffmpeg internal frames. Only available to ffmpeg filters and raw encoders. Avoids data copies between liquidsoap and ffmpeg.
• Copy content, that is encoded content stored as ffmpeg internal packets. Only available to the ffmpeg copy encoder and bitstream filters. Requires solid knowledge of media codecs and containers. Avoids transcoding by passing encoded data end-to-end inside liquidsoap scripts.

Enabling ffmpeg support

FFmpeg support is available via the external ocaml-ffmpeg binding package. Binary releases and Docker images include it by default.

If you are installing via opam, installing the ffmpeg package should do the trick:

% opam install ffmpeg

fdk-aac support in ffmpeg

Since 2.5.x, all binary releases and Docker images ship with a static FFmpeg build that includes fdk-aac.

For releases prior to 2.5.x, fdk-aac support requires the FFmpeg shared libraries compiled with libfdk-aac. On Debian, deb-multimedia.org provided such a build. See also this discussion.

Decoders

In most cases, the ffmpeg decoder requires no configuration. When enabled, it is the preferred decoder for all supported media. When using raw or copied content, it produces the required format automatically.

If you need to adjust its behavior:

settings.decoder.decoders controls which decoders are tried when decoding media files. Use it to restrict decoders, for instance to use only ffmpeg:

settings.decoder.decoders := ["FFMPEG"]

Priority for the decoder is set via:

settings.decoder.priorities.ffmpeg := 10

Use this to control whether ffmpeg is tried first, in combination with the other settings.decoder.priorities.* settings.

The settings.decoder.ffmpeg.codecs.* settings specify which decoder ffmpeg should use for each codec (there may be more than one available).

For instance, for the aac codec:

• settings.decoder.ffmpeg.codecs.aac.available() returns the list of available decoders, typically ["aac", "aac_fixed"].
• settings.decoder.ffmpeg.codecs.aac can be used to choose which decoder should be used, typically: settings.decoder.ffmpeg.codecs.aac := "aac"

When debugging issues with ffmpeg, it can be useful to increase the log verbosity.

settings.ffmpeg.log.verbosity := "warning"

This sets the verbosity of ffmpeg logs. Values from least to most verbose: "quiet", "panic", "fatal", "error", "warning", "info", "verbose", "debug", "trace".

By default, ffmpeg log messages are forwarded to liquidsoap's logger at the level set by settings.ffmpeg.log.level. To print them directly to standard error instead, disable log capture:

settings.ffmpeg.log.capture := false

Decoder arguments

In some cases, such as sending raw PCM data, the ffmpeg decoder needs additional arguments to know the format, codec, and other parameters.

There are two ways to provide them:

• For streams, the content_type argument can be used. The convention is to use "application/ffmpeg;<arguments>".
• For files, the ffmpeg_options metadata can be used, for instance using the annotate protocol: annotate:ffmpeg_options="<arguments>":/path/to/file.raw

Here is an example of an SRT input and output for sending raw PCM data between two instances:

Sender:

enc = %ffmpeg(
  format="s16le",
  %audio(
    codec="pcm_s16le",
    ac=2,
    ar=48000
  )
)

output.srt(enc, s)

Receiver:

s = input.srt(
  content_type="application/ffmpeg;format=s16le,ch_layout=stereo,sample_rate=48000"
)

If output.file were used instead, saving to bla.raw, the file can be read with a single source:

s = single("annotate:ffmpeg_options='format=s16le,ch_layout=stereo,sample_rate=44100':/tmp/bla.raw")

The same approach works with playlist or request.dynamic.

Encoders

See detailed ffmpeg encoders article.

Filters

See detailed ffmpeg filters article.

Subtitles

See the FFmpeg subtitles section of the subtitles page.

Multitrack

See the multitrack page to decode, process and encode sources with several audio, video or subtitle tracks.

Bitstream filters

FFmpeg bitstream filters modify the binary content of encoded data. They adjust codec and container aspects for specific uses, such as rtmp/flv output. They are particularly important for live switches on encoded content (see the Examples section).

All bitstream filters are listed in the FFmpeg documentation and our extra API reference. Here is an example:

% liquidsoap -h ffmpeg.filter.bitstream.h264_mp4toannexb

FFmpeg h264_mp4toannexb bitstream filter. See ffmpeg documentation for more
details.

Type: (?id : string?, source(video=ffmpeg.copy, 'a)) ->
source(video=ffmpeg.copy, 'a)

Category: Source / FFmpeg filter

Arguments:

 * id : string? (default: null)
     Force the value of the source ID.

 * (unlabeled) : source(video=ffmpeg.copy, 'a)

Methods:
...

Consult the FFmpeg documentation for details on what each filter does and how to use it.

Encoded data tweaks

Manipulating encoded content requires knowledge of the internal aspects of media codecs and containers. This section covers specific cases.

Relaxed copy content compatibility check

By default, liquidsoap tracks the content in ffmpeg.copy streams and only allows decoders to return strictly compatible content, such as matching video resolution or audio sample rate.

Some containers like mp4 allow streams where these parameters change between tracks. To permit this, relax the compatibility check:

settings.ffmpeg.content.copy.relaxed_compatibility_check := true

This is currently a global setting and may be refined per-stream in the future.

Shared encoders

Liquidsoap provides operators to encode data using %ffmpeg and share it across multiple outputs. This is called inline encoding. Here is an example:

audio_source = single(audio_url)
video_source = single(image)

stream = source.mux.video(video=video_source, audio_source)

stream = ffmpeg.encode.audio_video(
    %ffmpeg(
        %audio(codec="aac", b="128k"),
        %video(codec="libx264", b="4000k")
    ),
    stream
)

flv = %ffmpeg(
    format="flv",
    %audio.copy,
    %video.copy,
)

# Send to one youtube output:
output.youtube.live.rtmp(
    encoder = flv,
    stream,
    ...
)

mpegts = %ffmpeg(
    format="mpegts",
    %audio.copy,
    %video.copy,
)

# And to a hls one:
output.file.hls(
  ["mpegts", mpegts],
  stream,
  ...
)

Working with encoded data requires some knowledge of ffmpeg internals and media codecs and containers. In this example, the stream will have issues because the flv format requires global data, called extradata in ffmpeg terms.

When working with a single encoder such as:

%ffmpeg(
  format="flv",
  %audio(codec="aac", b="128k"),
  %video(codec="libx264", b="4000k")
)

When initializing the encoders, liquidsoap knows the target container is flv and implicitly enables the global header for each encoder.

With inline encoding, the target container is not known at encode time, and the encoded stream may be sent to multiple containers with different requirements.

There are two ways to solve this:

If all containers accept global headers, enable the flag in the encoder:

stream = ffmpeg.encode.audio_video(
    %ffmpeg(
        %audio(codec="aac", b="128k", flags="+global_header"),
        %video(codec="libx264", b="4000k", flags="+global_header")
    ),
    stream
)

If only one container needs global headers (flv here), apply the ffmpeg.filter.bitstream.extract_extradata bitstream filter to the stream sent to that container only:

audio_source = single(audio_url)
video_source = single(image)

stream = source.mux.video(video=video_source, audio_source)

stream = ffmpeg.encode.audio_video(
    %ffmpeg(
        %audio(codec="aac", b="128k"),
        %video(codec="libx264", b="4000k")
    ),
    stream
)

flv = %ffmpeg(
    format="flv",
    %audio.copy,
    %video.copy,
)

flv_stream = ffmpeg.filter.bitstream.extract_extradata(stream)

# Send to one youtube output:
output.youtube.live.rtmp(
    encoder = flv,
    flv_stream,
    ...
)

mpegts = %ffmpeg(
    format="mpegts",
    %audio.copy,
    %video.copy,
)

# And to a hls one:
output.file.hls(
  ["mpegts", mpegts],
  stream,
  ...
)

Examples

See detailed ffmpeg cookbook article.