We are delighted to announce the release of Liquidsoap 2.2.0! It is now available on our release page and should be available via opam shortly!
This release comes almost 4 months after the last stable release from the 2.1.x
release cycle and 14 months (!) after
forking the 2.2.x
release branch! It includes some exciting changes in track muxing/demuxing, HLS, sound processing,
and more. See below for a detailed list!
✨ New features
Here are the highlights:
🎛️ Multitrack
This is by far the biggest change in this release! This brings the ability to demux and remux tracks inside sources, making it possible to:
- Encode video with multiple audio tracks
- Create audio-only and audio/video streams from the same content, for instance, a regular audio stream and one with the studio’s video.
- Drop or specify which metadata or track marks track should be used.
- Apply specific audio effect or encoding to different tracks
And much more! The full documentation is here. We plan on expanding this support in the future in particular to allow track selection based on language, encoded content, etc.
🏷️ HLS metadata
At last! We now support metadata in HLS streams using a timed ID3 stream for the mpegts
container and plain ID3v2 tags
for adts
, mp3
, ac3
, and eac3
formats. There is currently no support for metadata with mp4
containers.
This feature is enabled by default, so you might want to check if it impacts your listeners before pushing it to
production. It can be turned off by setting id3
to false on your HLS streams.
Full documentation is here.
🎨 Colored logs
Small change but not the least important! Logs are now colored by default when printed on the console. This finally makes it possible to read logs with a high level of details!
We are aware of the need for more developer tooling and quality-of-life improvements! In the next release cycle, we hope to work on code formatting, highlighting, and more!
🕸️ New HTTP API
Interacting with your scripts is essential and, for this, web interfaces and APIs are really useful. In order to make our HTTP server easier to use, we wrote a new web API that is very close to node express’ API and should be fairly easy to use! The documentation is here
These changes also included a revamping of our SSL support which is now modular and with a new TLS optional support!
🎚️ Native stereotool support
While committed to open-source through and through, we also do want to meet our users where they are. To this end, it seems that a lot of them want to use the proprietary stereotool audio processing. Up until this version, the only option was via the external command line encoder and this was not satisfactory.
With this release, it is now possible to use the shared library distributed by the author, which provides support for a
new stereotool
internal operator that is much easier to integrate. See the documentation
here.
📟 Records enhancements
As part of the language changes required for multitrack, we now support the following operations on records:
Record spread:
let {foo, bar, ...rest} = someRecord
let otherRecord = { bla = "blo", ...someRecord }
Additionally, we now support optional record methods, for instance:
def f(x) =
if x?.foo == null() then
print("x does not have method foo")
else
print("x has method foo")
end
end
🪢 Support for YAML parsing/rendering
Following the recent JSON parsing feature, we now support YAML parsing in a very similar way as json.
🔮 Memory optimization
While we are aware that memory consumption with this release may have increased a bit due to ongoing changes, we have done our best to introduce more ways to control it and understand its usage.
In particular, we now support the alternative jemalloc
memory allocator, enabled in all our release assets and
configurable via the internal settings.
We also introduced two new audio content formats, pcm_s16
and pcm_f32
that can be used to store audio internally as,
resp., 16-bit signed integers or 32-bit floating point numbers. Our default internal format is OCaml’s native 64-bit
floating point numbers.
We also added a new track.audio.defer
operator that can be used to buffer large amount of audio data without impacting
performances.
You can read more about memory utilization in liquidsoap here.
🐪 Switch to dune
and javascript runtime
While perhaps more exciting to developers, the project has now fully moved to the OCaml dune
builder. This provides an
extra level of flexibility, In particular, we were able to extract the code that is specific to the liquidsoap language,
that is everything that pertains to parsing/evaluating/type checking without the streaming and system-specific operators
and export it as an online playground. We’re not sure yet what we’ll do with it. It
might be possible, for instance, to write a javascript backend to use liquidsoap scripts with the
Web media APIs!
🕵️ Versioning and release assets
For a little over a year now, we have switched to rolling release cycles with maintenance and bugfixes applying only
to the current release cycle. Regular releases are tagged vX.Y.Z
(for instance v2.2.0
) on GitHub and docker while
ongoing releases are tagged rolling-release-vX.Y.Z
.
When an initial release, for instance, 2.2.0
, is being worked on, bugfixes and user issues are addressed for users
using the 2.1.x
releases. While we would like to extend support, this is the best that we can do with our limited
resources!
rolling-release-vX.Y.Z
tags denote the releases being worked on. For stable release branches, this would be for
instance, rolling-release-v2.2.1
after release v2.2.0
. For a yet-to-be released initial version, this would be for
instance rolling-release-v2.3.x
. We try our best to make those releases as bug-free as possible. Using one of them to
test your production script also guarantees the fastest response to bugs and issues!
For release assets, we try to maintain two packages for debian and ubuntu distributions, one with the latest LTS or
stable and one with a recent release. The new minimal
assets are, as the name suggests, minimal builds. They contain
a limited set of features and standard library operators. Minimal builds are intended for most production run and should
limit the risk of issues such as segfault, etc. If your script can run with it, we recommend it over the fully featured
builds.
For each release asset, you can consult the associated .config
file to see which features are enabled.
Docker release images are located at: savonet/liquidsoap:v2.2.0
. The release tag may be updated if needed. You can use
git sha-based images to pick a fixed build, e.g. savonet/liquidsoap:<sha>
Lastly, we may update the list of release assets on the GitHub release page. If you are looking for a permanent release asset links make sure to check out savonet/liquidsoap-release-assets.
🧮 Migration guide
We listed most of the migration issues you might run into on this page. The detailed changelog below may also help.
As a reminder, we strongly recommend testing your script in a staging environment, even between minor releases, to make sure that everything is working correctly before pushing a new liquidsoap version to production!
📖 Changelog
The full shebang!
2.2.0 (2023-07-21)
New:
- Added support for less memory hungry audio formats, namely
pcm_s16
andpcm_f32
(#3008). - Added support for native osc library (#2426, #2480).
- SRT: added support for the
passphrase
,pbkeylen
, andstreamid
; added native type for srt sockets with methods; moved stats to socket methods, addedsocket()
method on srt input/outputs (#2556). - HLS: added support for ID3 in-stream metadata (#3154) and custom tags (#2898).
- Added support for FLAC metadata (#2952).
- Added support for YAML parsing and rendering (#2855).
- Added support for the proprietary shared stereotool library (#2953).
- Added TLS support via
ocaml-tls
(#3074). - Added
video.align
. - Added
string.index
. - Added support for ffmpeg decoder parameters to allow decoding of raw PCM stream and file (#3066).
- Added support for unit interactive variables: those call a handler when their value is set.
- Added support for id3v2
v2.2.0
frames and pictures. - Added
track.audio.defer
to be used to buffer large amount of audio data (#3136). - Added
runtime.locale.force
to force the system’s locale (#3231). - Added support for customizable, optimized
jemalloc
memory allocator (#3170). - Added
source.drop
to animate a source as fast as possible. - Added in-house replaygain computation:
source.replaygain.compute
to compute the replaygain of a source.file.replaygain
to compute the replaygain of a file.
- Added support for ImageLib to decode images.
- Added support for completion in emacs based on company (#2652).
- Added syntactic sugar for record spread:
let {foo, gni, ..y} = x
andy = { foo = 123, gni = "aabb", ...x}
(#2737). - Added
file.{copy, move}
(#2771). - Detect functions defining multiple arguments with the same label (#2823).
- Added
null.map
. - References of type
'a
are now objects of type(()->'a).{set : ('a) -> unit}
. This means that you should usex()
instead of!x
to get the value of a reference. Setting a reference can be done both byx.set(v)
andx := v
, which is still supported as a notation (#2881). - Added
ref.make
andref.map
. - Added
video.board
,video.graph
,video.info
(#2886). - Added the
pico2wave
protocol to perform speech synthesis using Pico TTS (#2934). - Added
settings.protocol.gtts.lang
to be able to selectgtts
’ language, addedsettings.protocol.gtts.options
to be able to add any other option (#3182). - Added
settings.protocol.pico2wave.lang
to be able to selectpico2wav
language (#3182). - Added
"metadata_url"
to the default list of exported metadata (#2946). - Added log colors!
- Added
list.filter_map
andlist.flatten
. - Added
medialib
to store metadata of files in a folder and query them (#3115). - Added
--unsafe
option (#3113). This makes the startup much faster but disables some guarantees (and might even make the script crash…). - Added
string.split.first
(#3146). - Added
string.getter.single
(#3125).
Changed:
- Switched to
dune
for building the binary and libraries. - Changed
cry
to be a required dependency. - Changed default character encoding in
output.harbor
,output.icecast
output.shoutcast
toUTF-8
(#2704). - BREAKING: all
timeout
settings and parameters are nowfloat
values and in seconds (#2809). - BREAKING: in
output.{shoutcast,icecast}
:- Old
icy_metadata
was renamed tosend_icy_metadata
and changed to a nullablebool
.null
means guess. - New
icy_metadata
now returns a list of metadata to send with ICY updates. - Added
icy_song
argument to generate default"song"
metadata for ICY updates. Defaults to<artist> - <title>
when available, otherwiseartist
ortitle
if available, otherwisenull
, meaning don’t add the metadata. - Cleanup, removed parameters that were irrelevant to each operator, i.e.
icy_id
inoutput.icecast
and etc. - Make
mount
mandatory andname
nullable. Usemount
asname
whenname
isnull
.
- Old
reopen_on_error
andreopen_on_metadata
inoutput.file
and related operators are now callbacks to allow dynamic handling.- Added the
reopen
method tooutput.file
. - Added support for a Javascript build and interpreter.
- Removed support for
%define
variables, superseded by support for actual variables in encoders. - Cancel pending append when skipping the current track on the
append
source. - Errors now report proper stack trace via their
trace
method, making it possible to programmatically point to file, line, and character offsets of each step in the error call trace (#2712). - Reimplemented
harbor
http handler API to be more flexible. Added a new node/express-like registration and middleware API (#2599). - Switched default persistence for cross and fade-related overrides to follow documented behavior. By default,
"liq_fade_out"
,"liq_fade_skip"
,"liq_fade_in"
,"liq_cross_duration"
and"liq_fade_type"
now all reset on new tracks. Usepersist_overrides
to revert to the previous behavior (persist_override
forcross
/crossfade
) (#2488). - Allow running as root by default when the docker container can be detected using the presence of a
/.dockerenv
file. id3v2
argument of the%mp3
encoder was changed to"none"
or version number to allow to choose the metadata version.true
is still accepted and defaults to version3
. Switched to our internal implementation so that it does not requiretaglib
anymore.- Moved HLS outputs stream info as optional methods on their respective encoder.
- Changed
self_sync
ininput.ffmpeg
to be a boolean getter, changedself_sync
ininput.http
to be a nullable boolean getter. Setself_sync
totrue
ininput.http
when an icecast or shoutcast server can be detected. - Add the
sorted
option tofile.ls
. - Add the
buffer_length
method toinput.external.rawaudio
andinput.external.wav
(#2612). - Added full
OCaml
backtrace astrace
to runtime errors returned from OCaml code. - Removed confusing
let json.stringify
in favor ofjson.stringify()
. - Font, font size, and colors are now getters for text operators (
video.text
,video.add_text
, etc.) (#2623). - Add the
on_cycle
option tovideo.add_text
to register a handler when cycling (#2621). - Renamed
{get,set}env
intoenvironment.{get,set}
. - Renamed
add_decoder
,add_oblivious_decoder
andadd_metadata_resolver
into, respectively,decoder.add
,decoder.oblivious.add
,decoder.metadata.add
. - Deprecated
get_mime
, addedfile.mime.libmagic
andfile.mime.cli
, madefile.mime
tryfile.mime.libmagic
if present andfile.mime.cli
otherwise, changed returned value when no mime was found tonull()
. - Return a nullable float in
request.duration
. - Removed
--list-plugins-json
and--list-plugins-xml
options. - Added
--list-functions-json
option. - Removed built-in use of
strftime
conversions in output filenames, replaced by an explicit call totime.string
(#2593). - Added nullable default to
{int,float,bool}_of_string
conversion functions, raise an exception if conversion fails and no default is given. - Deprecated
string_of
in favor ofstring
(#2700). - Deprecated
string_of_float
in favor ofstring.float
(#2700). - Added
settings.protocol.youtube_dl.timeout
to specify a timeout when using theyoutube-dl
protocol (#2827). Useyt-dlp
as the default binary for the protocol. - The
sleeper
operator is now scripted (#2899). - Reworked remote request file extension resolution (#2947).
- REMOVED
osx-secure-transport
. Doubt it was ever used, API was deprecated upstream (#3067). - Renamed
rectangle
toadd_rectangle
, and similarly forline
.
Fixed:
- The randomization function
list.shuffle
used inplaylist
was incorrect and could lead to incorrectly randomized playlists (#2507, #2500). - Fixed srt output in listener mode to allow more than one listener at a time and prevent the listening socket from being re-created on listener disconnection (#2556).
- Fixed race condition when switching
input.ffmpeg
-based urls (#2956). - Fixed deadlock in the
%external
encoder (#3029). - Fixed crash in encoders due to concurrent access (#3064).
- Fixed long-term connection issues with SSL (#3067).