Babeltrace 2.0.1 - Amqui

Today we're releasing the first patch-level release of Babeltrace 2.0 "Amqui".

Amqui (/ɒmkwiː/) is a town in eastern Québec, Canada, at the base of the Gaspé peninsula in Bas-Saint-Laurent.

Located at the confluence of the Humqui and Matapédia Rivers, its proximity to woodlands makes it a great destination for outdoor activities such as camping, hiking, and mountain biking.

What's new since Babeltrace 2.0.0?

This release contains minor bug fixes which are mainly related to packaging and documentation issues.

Of note for packagers, we have changed the autoconf package name to babeltrace2. This fixes co-installation problems with Babeltrace 1.x, but also changes the naming scheme of the official tarball.

The official tarballs of Babeltrace 2 are now named babeltrace2-$VERSION.tar.bz2. We have added a symlink to the original Babeltrace 2.0.0 release to reflect this new naming scheme.

The full change log follows.

Babeltrace 2.0.1

e3757f6 fix: Remove empty python bindings documentation
f1f3b04 README: Babeltrace 2 was released in 2020
f1598eb fix: set autoconf package name to babeltrace2
fe5ee42 Typo: occured -> occurred
92da26a .gitignore: Add missing bt2/native_bt.d
bee055d fix: build failure on ppc64el with '-Werror=format-overflow='
2f65757 Silence -Wnull-dereference warning in generated CTF parser code
04ab8e4 plugin-so.c: add comment about why we're not using a GLib linked list
c0c5d93 doc: graph.h: do not link to man7.org for babeltrace(1)
e7ced9c fix: common/list.h is LGPL-2.1
531c3c7 fix: build Python bindings with GCC10
003f30d fix: use correct function to print 'enum bt_ctf_scope'
3302c66 Update working version to Babeltrace 2.0.1

Babeltrace 2.0.0 - Amqui

Hello people!

This is it: the day you’ve been waiting for has finally arrived.

Today we’re releasing Babeltrace 2.0 “Amqui”! 🥳

Amqui (/ɒmkwiː/) is a town in eastern Québec, Canada, at the base of the Gaspé peninsula in Bas-Saint-Laurent.

Located at the confluence of the Humqui and Matapédia Rivers, its proximity to woodlands makes it a great destination for outdoor activities such as camping, hiking, and mountain biking.

By the way, Babeltrace has a new dedicated domain name: babeltrace.org.

Background

Just a few words about why Babeltrace 2 took about five years to complete.

First, let us clarify that from day one, one of the most important imperatives of the project was long-term stability: we wanted Babeltrace 2 to last for a long time and for its library’s API to become a well-known reference in the trace processing territory, a bit like Libxml2 has had major version 2 for about 20 years. All the other Babeltrace 2 parts depend on the library’s C API: the command-line interface (CLI), the plugins, and the Python bindings.

So, as we needed to modify the library, CLI, and plugins to adapt to new requirements during the development phase, we postponed the release so as not to have to release Babeltrace 3 a year after and Babeltrace 4 another year after.

Second, the scope and requirements of the project evolved. Because there were at most two developers working on Babeltrace 2 at the same time during its first years, the project took enough time to develop for new ideas to emerge and translate into changes, sometimes significant, to the project’s modules.

For example, the very first requirement was for the libbabeltrace2 API to use the same objects as Babeltrace 1’s CTF writer API. We put a lot of effort backfitting new concepts into the CTF writer API and making sure not to break existing user code. But in the end, we realized that this condition was too limiting and that we had to split the APIs completely to have the freedom to introduce all the features we wanted.

When the API finally met our needs feature-wise in 2018, we measured and found that a simple CTF-to-text conversion task with Babeltrace 2 was approximately 23 times slower than the equivalent with Babeltrace 1! Hence we entered an optimization phase in which we had to redefine core concepts of the library (for example, introduce unique objects to create object pools) so that, today, Babeltrace 2’s performance is comparable to Babeltrace 1’s.

Revamping the C API to optimize the library means we had to readapt the CLI, plugins, and Python bindings. We spent the beginning of 2019 synchronizing the project’s parts, but also improving and uplifting the test suite and fixing a lot of lingering bugs.

The last few months were invested in writing a complete C API documentation and in more bug fixes, tests, and build system improvements.

Today ends this journey to Babeltrace 2.0. It’s been a long one, but we made it. All the next releases of Babeltrace 2 will be minor ones for the foreseeable future, never breaking the library’s API.

Babeltrace 2 is:

• 120,000 lines of C.

• 7000 lines of Python.

• 21,000 tests.

• 30,000 lines of C API documentation.

• 24 manual pages

We hope you enjoy using our software as much as we enjoyed making and refining it over the last years.

What’s new since Babeltrace 1?

The Babeltrace 2 project is completely independent from Babeltrace 1. In fact, you can install both on the same system as they share no common files. We only made sure to keep the babeltrace2 CLI tool backward compatible with the original babeltrace tool.

See babeltrace2-intro(7) to learn more about Babeltrace 2.

• General

  • Full plugin support: any user can distribute a Babeltrace 2 plugin and, as long as libbabeltrace2 finds it, any application linked to libbabeltrace2 can load it and use it.

    Plugins are not just trace format encoders and decoders: they package source, filter, and sink component classes so that you can connect specialized, reusable components together in a trace processing graph to create a customized trace conversion or analysis device.

    This modular strategy is much like how the FFmpeg, GStreamer, and DirectShow projects approach media stream processing.

  • All the parts of the Babeltrace 2 project run on the major operating systems, including Linux, macOS, and Windows.

  • Some component classes, such as sink.text.pretty (similar to the text output format of the babeltrace CLI) and sink.text.details, can write color codes to the standard output when it’s connected to a color-enabled terminal.

    The Babeltrace 2 log, printed to the standard output, can also be colorized.

• Command-line interface

  • Whereas you can convert traces from one format to another with Babeltrace 1’s CLI tool, babeltrace, you can also execute a custom trace manipulation task with babeltrace2 thanks to its run command.

  • The convert command features an automatic source component discovery algorithm to find the best suited components to create for a given non-option argument (file or directory path, or custom string like an LTTng live URL).

    For example:

    $ babeltrace2 /path/to/ctf/trace
    
    $ babeltrace2 net://localhost/host/myhost/my-session
    

• CTF input/output

  • The source.ctf.fs component class, which is more or less the equivalent of Babeltrace 1’s ctf input format, has features not found in Babeltrace 1:

    • The component handles many trace quirks which are the results of known tracer bugs (which are fixed today) and corner cases in LTTng-UST, LTTng-modules, and barectf, making it possible to decode malformed packets.

    • The component merges CTF traces sharing the same UUID into a single, logical trace.

      This feature supports LTTng 2.11’s tracing session rotation trace chunks.

  • With a sink.ctf.fs component, you can create CTF traces on the file system.

    With the babeltrace2 tool, you can use the --output-format=ctf and --output options to create an implicit sink.ctf.fs component.

    For example:

    $ babeltrace2 /path/to/input/trace --output-format=ctf --output=trace-dir
    

• LTTng live input
  Babeltrace 1’s babeltrace command exits successfully when it cannot find an LTTng live (--input-format=lttng-live option) tracing session.

  The session-not-found-action initialization parameter controls what a source.ctf.lttng-live message iterator does when it cannot find the remote tracing session:

  • If the action is end, the message iterator does like babeltrace and simply ends successfully.

  • If the action is continue (the default), the message iterator never ends: it keeps on trying until the tracing session exists, indeed “subscribing” to the session.

• Library

  • libbabeltrace2 shares nothing with libbabeltrace.

    The Babeltrace 2 library C API has features such as:

    • A single header file.

    • Function precondition and postcondition checking.

    • Object-oriented model with shared and unique objects.

    • Strict C typing and const correctness.

    • User-extensible classes.

    • Rich, thread-safe error reporting.

    • Per-component and per-subsystem logging levels.

    • Trace intermediate representation (IR) objects to make the API trace-format-agnostic.

    • A versioned protocol for the exchange of messages...

Babeltrace 2.0.0 - Release Candidate 4

Hi everyone!

Two weeks after having released Babeltrace 2’s second release candidate, today we’re releasing Babeltrace 2.0.0-rc4.

Note

Due to human error, Babeltrace 2.0.0-rc3 is a malformed release. Babeltrace 2.0.0-rc3 and Babeltrace 2.0.0-rc4 have the same changes since Babeltrace 2.0.0-rc2.

What’s new since Babeltrace 2.0.0-rc2?

Improvements

• General

  • Introduce the BABELTRACE_EXEC_ON_ABORT environment variable.

    BABELTRACE_EXEC_ON_ABORT can contain a shell command line to execute when any part of the Babeltrace project aborts (for example, when an internal assertion fails) before the process exits.

    See 8403432f.

  • Add precondition assertions to verify that the current thread has no error when you call a libbabeltrace2 function (except reference count functions).

    See 7c7324d3.

  • Add postcondition assertions to verify that the user functions which libbabeltrace2 calls return an error status code when the current thread has an error.

    See d6f6a5aa.

  • Emit dedicated bright terminal color codes if supported by the terminal emulator.

    This makes the babeltrace2 CLI tool and some project plugins emit correct bright color codes for the kitty terminal emulator, for example.

    You can override this behaviour with the new BABELTRACE_TERM_COLOR_BRIGHT_MEANS_BOLD environment variable.

    See 4253e1eb.

  • In libbabeltrace2’s API: make the bt_value_map_foreach_entry_func and bt_value_map_foreach_entry_const_func function types return a status code instead of bt_bool.

    This makes it possible for the user function to append a cause to the current thread’s error and return an error status code. When the user function returns a general error status code, bt_value_map_foreach_entry() returns BT_VALUE_MAP_FOREACH_ENTRY_STATUS_USER_ERROR (and an equivalent code for bt_value_map_foreach_entry_const()).

    The new way to interrupt the outer loop from the user function is to return BT_VALUE_MAP_FOREACH_ENTRY_FUNC_STATUS_INTERRUPT.

    See 0589c646.

• Command-line interface
  Exit with status 2 when the process receives the SIGINT signal.

  This makes it possible to distinguish a successful execution (status 0) from an execution that ends because of an error (status 1).

  See 851802b1.

• source.ctf.fs component class

  • Check the version of LTTng index files to avoid reading files of future backwards-incompatible versions.

    See 2068d50b.

  • If the same packets exist in different data stream files of a given CTF trace, read each of them only once.

    This feature makes it possible to open overlapping LTTng snapshots without emitting the event and packet messages twice or more like Babeltrace 1 does.

    For example:

    $ babeltrace2 snapshot1 snapshot2 snapshot3
    
    $ babeltrace2 snapshot-directory
    

    See 148f02cb.

• source.ctf.lttng-live component class
  Return BT_COMPONENT_CLASS_MESSAGE_ITERATOR_NEXT_METHOD_STATUS_AGAIN when a signal interrupts an I/O system call and the graph’s interrupter is set; restart the system call otherwise.

  Because the babeltrace2 CLI tool sets the graph’s interrupter when it receives the SIGINT signal, this change makes it possible to exit with status 2 (instead of status 1, an error) when it receives the SIGINT signal and a source.ctf.lttng-live message iterator is currently being executed.

  See 8dd99d00.

• filter.utils.muxer component class
  Append causes to the current thread’s error when encountering some error conditions.

  See f331468c.

Notable bug fixes

• Simple sink component class
  Append error causes only when returning an error status code.

  See eb657c7b.

Upcoming

We are still hard at work putting the finishing touches on our way to the final 2.0.0 release.

We invite you to try this release candidate and report any problems you may encounter to the [email protected] mailing list or through the Babeltrace bug tracker.

Documentation

We are currently documenting the entire Babeltrace 2 C API to make the development of new component classes as easy as possible.

We’ll also work on the Python bindings documentation.

The documentation of the API will be made available on the official Babeltrace website.

Other tasks

• Improve test coverage.

• Improve resilience to corrupted/malformed CTF traces.

• Minor internal cleanups and bug fixes.

Important links

• Babeltrace 2.0.0-rc4 tarball

• Babeltrace website

• Mailing list for support and development: [email protected]

• IRC channel: #lttng on irc.oftc.net

• Git repository

• GitHub project

• Continuous integration

• Code review

Babeltrace 2.0.0 - Release Candidate 2

Hi everyone!

Two weeks after having released Babeltrace 2’s first release candidate, today we’re releasing Babeltrace 2.0.0-rc2.

What’s new since Babeltrace 2.0.0-rc1?

Improvements

General

• Introduce BT_ASSERT_DBG() for assertions that must only be evaluated in debugging mode (when the BABELTRACE_DEBUG_MODE environment variable is set to 1 at configuration time).

  BT_ASSERT() assertions are now always evaluated, regardless of the debugging/production mode.

  This change’s purpose is to keep as many slow-path assertions as possible to detect more Babeltrace 2 programming errors, even in production.

  See 98b15851.

• Remove ABI versioning in Babeltrace 2 shared object plugins as we don’t need this property.

  See 7c0244d6.

• In libbabeltrace2’s API:

  • Rename BT_RANGE_SET_ prefixes to BT_INTEGER_RANGE_SET_.

    See eb171ee9.

  • Make bt_version_get_major(), bt_version_get_minor(), and bt_version_get_patch() functions return unsigned int instead of int.

    See 78deb913.

  • Remove the BT_GRAPH_RUN_STATUS_END status code: bt_graph_run() now returns BT_GRAPH_RUN_STATUS_OK when all the graph’s sink components are ended.

    See9669d693

ctf plugin

Append error causes at more source locations.

This means more precise error messages reported by the babeltrace2 CLI tool, for example.

See b7d2695a, b30284d1, 2246e99da, and 1419db2bc.

source.ctf.fs component class

Return an error when failing to load or create an index for a trace.

See 29a8227a.

filter.lttng-utils.debug-info component class

Create stream classes and event classes which correspond to stream classes and event classes in incoming trace classes as soon as we encounter a new trace class, rather than as needed.

See db5d746d.

Python bindings

In bt2.Graph.add_component(), validate that the params parameter is implicitly convertible to a bt2.Mapvalue object (dict, for example).

See 401b7022.

Bug fixes

General

Fix many issues reported by static analysis tools (scan-build, Coverity, and compiler warnings).

source.ctf.fs component class

Fix an index handling bug which occured when multiple data stream files in a CTF trace belong to the same data stream.

See ce75de14.

source.ctf.lttng-live component class

Fix a bug in which a disappearing LTTng live tracing session could lead to an assertion failure.

See ba90bce7.

Python bindings

Fix an assertion failure which occurs when a trace or trace class destruction listener raises an exception.

See 64961f8b.

Upcoming

We are still hard at work putting the finishing touches on our way to the final 2.0.0 release.

We invite you to try this release candidate and report any problems you may encounter to the [email protected] mailing list or through the Babeltrace bug tracker.

Documentation

We are currently documenting the entire Babeltrace 2 C API to make the development of new component classes as easy as possible.

We’ll also work on the Python bindings documentation.

The documentation of the API will be made available on the official Babeltrace website.

Other tasks

• Improve test coverage.

• Improve resilience to corrupted/malformed CTF traces.

• Minor internal cleanups and bug fixes.

Important links

• Babeltrace 2.0.0-rc2 tarball

• Babeltrace website

• Mailing list for support and development: [email protected]

• IRC channel: #lttng on irc.oftc.net

• Git repository

• GitHub project

• Continuous integration

• Code review

Babeltrace 2.0.0 - Release Candidate 1

Hi everyone!

We’re very excited to announce the first Babeltrace 2.0 release candidate! 😃

What’s new since Babeltrace 2.0.0-pre5?

Babeltrace 2.0.0-pre5 was released 3 May 2019.

We brought many improvements to the CLI, the project plugins, and the library and its Python bindings.

We also updated all the Babeltrace 2 manual pages.

The build-time and run-time requirements for Babeltrace 2.0.0-rc1 are updated:

• Babeltrace 2.0.0-rc1 does not need the popt library anymore to parse command-line arguments.

  We wrote our own command-line argument parser to remove this obsolete requirement and to support advanced, position-dependent parsing.

• Babeltrace 2.0.0-rc1 requires GLib ≥ 2.28 at build time and run time instead of the previous ≥ 2.22 requirement.

Command-line interface

Automatic source component discovery

With the convert command’s new automatic source component discovery feature, specify a path, URI, or arbitrary string to let babeltrace2 find the most appropriate components to use:

$ babeltrace2 /path/to/ctf/trace
$ babeltrace2 net://localhost/host/myhost/my-session

This means you don’t need to create an explicit component with the --component option most of the time.

Behind the scenes, this feature queries the babeltrace.support-info object from all known component classes: a component class can reply with a confidence score to indicate its ability to handle a given path or custom string.

For example, a source.ctf.fs (CTF trace directory source) component class replies to this query whether or not the given path looks like a CTF trace directory.

NOTE: Source component classes do not have to implement the babeltrace.support-info query.

If a component class does not implement this query, the automatic source component discovery algorithm won’t consider it. In that case, you must instantiate such a component class explicitly with the --component option.

Plugins

New sink.text.details component class

This new component pretty-prints all the details of the messages it consumes.

The output is optimized to be human-readable, yet it is deterministic for testing and debugging purposes.

You can use a sink.text.details component to easily inspect messages flowing at any point of a graph and make it easy to test and troubleshoot other component classes.

The Babeltrace 2 project uses this component class for many tests.

source.ctf.fs

A source.ctf.fs component can now open and read malformed CTF traces produced by tracers having known bugs and corner cases (LTTng-UST, LTTng-modules, and barectf).

For example, a source.ctf.fs component can now read data streams which the lttng-crash tool generates, even if the last packets can be malformed.

See the Trace quirks section of the babeltrace2-source.ctf.fs(7) manual page for more details.

Library and Python bindings

Library-specific: simple sink component

Add a sink component to a graph easily using the new simple sink component C API.

A simple sink component is a sink component with a single input port and a single message iterator.

This interface reduces the amount of code you need to write to create a basic Babeltrace 2 message consumer.

Just provide a consumption user function which receives the component’s message iterator to consume upstream messages.

You can also implement other user functions:

Initialization function

Called once during the initialization phase of the graph.

Finalization function

Called once during the finalization phase of the graph.

Python-specific: automatic source component discovery using bt2.TraceCollectionMessageIterator

Integrate the same automatic source component discovery mechanism which the babeltrace2 CLI’s convert command uses to a Python application using a bt2.TraceCollectionMessageIterator object.

With a bt2.TraceCollectionMessageIterator object, your application can access additional component classes installed on the system from the get go.

New field class types

New field classes are available in libbabeltrace2 and its Python bindings to support eventual, new CTF 2 field classes.

The new fields are:

Boolean

Contains a boolean value.

Bit array

Contains a fixed-size array of bits.

A bit array field can represent a CPU status or flag register value, for example.

Unlike integer fields, a bit array field is not a numerical field.

Option

Contains an optional field.

An option field class is conceptually equivalent to a variant field class with two options: no field class and the optional field class.

User attributes

Attach custom user attributes to any of the following trace IR objects:

• Clock class

• Event class

• Field class

• Stream

• Stream class

• Trace

• Trace class

This new property exists because we expect the CTF 2 metadata objects to contain user attributes too.

Error reporting

Report rich error causes from user functions with the new thread-safe error reporting API.

The error reporting API makes it possible for the various actors interacting through libbabeltrace2 to precisely describe the chain of events that lead to a given error. As a library user, you can access this list of causes and reduce the time needed to troubleshoot problems.

This feature is similar to the stack trace which exception objects contain in many programming languages.

When an error occurs, the babeltrace2 CLI uses this error reporting API to show the top-level error itself, but also a detailed list of its causes, for example:

ERROR:    [Babeltrace CLI] (babeltrace2.c:2546)
  Graph failed to complete successfully
CAUSED BY [Babeltrace library] (graph.c:604)
  Component's "consume" method failed: ..., comp-name="sink.text.details", ...
CAUSED BY [Babeltrace library] (iterator.c:889)
  Component input port message iterator's "next" method failed: ..., iter-upstream-comp-class-name="debug-info", ...
CAUSED BY [Babeltrace library] (iterator.c:889)
  Component input port message iterator's "next" method failed: ..., iter-upstream-comp-name="muxer", ...
CAUSED BY [Babeltrace library] (iterator.c:889)
  Component input port message iterator's "next" method failed: ..., iter-upstream-comp-class-name="MyFirstSource", ...
CAUSED BY [src.demo.MyFirstSource (some-name): 'source.demo.MyFirstSource'] (bt2/native_bt_log_and_append_error.h:100)
  Traceback (most recent call last):
    File "/usr/local/lib/python3.6/dist-packages/bt2/message_iterator.py", line 151, in _bt_next_from_native
      msg = next(self)
    File "./bt_plugin_foo.py", line 33, in __next__
      raise Exception
  Exception

In the Python bindings, raise any exception object as usual from a user function: the bt2 package converts it to a libbabeltrace2 error cause for other parties.

Coinstallation with Babeltrace 1

To make the transition from Babeltrace 1 to Babeltrace 2 easier, we decided to make both projects coinstallable thanks to those changes:

• The Babeltrace 2 CLI tool is named babeltrace2.

• The Babeltrace 2 library’s include directory is named babeltrace2.

• The Babeltrace 2 library is named libbabeltrace2.

• The Babeltrace 2 manual page names start with babeltrace2-.

Distribution packages will be available for both Babeltrace 1 and Babeltrace 2.

Message Interchange Protocol versioning

Future-proof component classes using Message Interchange Protocol versioning.

As Babeltrace 2 evolves, we expect to introduce new concepts and objects to libbabeltrace2 such as field class types, message types, methods, and message ordering requirements. To ensure forward and backward compatibility can be maintained across future releases of Babeltrace 2, Babeltrace 2.0.0-rc1 adds the concept of Message Interchange Protocol (MIP).

Should we introduce a breaking change to the protocol which trace processing graph components must honor, we’ll bump the MIP version to announce the modified interface.

You can consider the MIP version as an implicit precondition variable for almost all the libbabeltrace2 functions.

Given the expected initialization parameters, a component class can report a range of supported MIP versions. You can then build a graph which operates with a specific MIP version to ensure all its components exchange messages using the same protocol.

const correctness

The Babeltrace 2 C API uses the const qualifier liberally to catch many programming errors at compile time.

As the Python language does not have a const feature to express immutability co...

v2.0.0-pre5

What's new since pre4?

New features

Trace merging

It is now possible to merge CTF traces that share the same UUID to a
single CTF trace. This is especially useful when combined with the
tracing session rotation feature introduced by LTTng 2.11.

For instance, it is possible to combine an arbitrary number of trace
archives and merge them back together.

Example:

A tracing session configured to automatically rotate every two seconds
will result in a large number of independent traces, as follows:

/path/to/my_session-20190501-181356
└──  archives
    ├──  20190501T181356-0400-20190501T181358-0400-1
    │   └──  ust
    │       └──  uid
    │           └──  1000
    │               └──  64-bit
    │                   ├──  channel0_0
    │                   ├──  channel0_1
    │                   ├──  channel0_2
    │                   ├──  channel0_3
    │                   ├──  index
    │                   └──  metadata
    ├──  20190501T181358-0400-20190501T181400-0400-2
    │   └──  ust
    │       └──  uid
    │           └──  1000
    │               └──  64-bit
    │                   ├──  channel0_0
    │                   ├──  channel0_1
    │                   ├──  channel0_2
    │                   ├──  channel0_3
    │                   ├──  index
    │                   └──  metadata
    ├──  20190501T181400-0400-20190501T181402-0400-3
    │   └──  ust
    │       └──  uid
    │           └──  1000
    │               └──  64-bit
    │                   ├──  channel0_0
    │                   ├──  channel0_1
    │                   ├──  channel0_2
    │                   ├──  channel0_3
    │                   ├──  index
    │                   └──  metadata
    ...

With this feature, it is now possible to combine all, or a subset, of
the archived trace chunks. The following example combines two of the
archived trace chunks into a single trace.

babeltrace [...]/archives/20190501T181358-0400-20190501T181400-0400-2 \
           [...]archives/20190501T181402-0400-20190501T181404-0400-4 \
           --output-format=ctf --output /tmp/merged-trace

This command produces the following trace:

/tmp/merged-trace
└──  myhostname
    └──  my_session-20190501T181356-0400
        └──  ust
            └──  uid
                └──  1000
                    └──  64-bit
                        ├──  channel0_0
                        ├──  channel0_1
                        ├──  channel0_2
                        ├──  channel0_3
                        └──  metadata

Note that this command uses new trace environment fields which will be
produced by the final release of LTTng 2.11. In the meantime, using
the existing LTTng 2.11-rc1 release will result in a different output
trace hierarchy.

LTTng live "subscribe" mode

It is now possible to configure the src.ctf.lttng-live component in
"subscribe" mode. In this mode, the source will attempt to consume an
LTTng relay daemon's session and retry periodically if the session
does not exist.

This is in contrast with the default behaviour which will return (with
success) immediately if the session to consume does not exist on the
LTTng relay daemon.

The following command demonstrates the use of this new mode:

babeltrace --component=src.ctf.lttng-live \
           --url=net://relayhost/host/tgthost/my-session \
           --params='session-not-found-action="continue"'

Note that we are planning the addition of a dedicated option to the CLI by the
time of the final release.

CLI support for array component parameters (--params)

Users of the CLI can now use array parameters when configuring a
component. See the following example:

babeltrace --component=src.my.class \
           --params='tokens=[23, 17, [42, "meow"]], path="/path/to/trace"'

CTF source can now recursively search paths (and path arrays)

Using the feature above, it is now possible to provide a list of paths
to the src.ctf.fs component and rely on the component to recursively
open CTF traces it finds.

See the following example:

babeltrace --component=src.ctf.fs \
           --params='paths=["trace1", "trace2", "trace3"]'

Performance improvements

One of the primary focus areas of the pre5 release was optimization
work to bring Babeltrace 2's CTF reader back to performance parity
with Babeltrace 1.x and reduce the overhead of the graph processing
infrastructure. As of this release, our benchmarks indicate that
Babeltrace 2 is now slightly faster (~4%) than the latest Babeltrace 1
release with a dummy output.

This work (and performance comparison) was the subject of a talk by
Philippe Proulx at the latest Tracing Summit. The video [1] and slides
[2] of this presentation are available and provide the rationale
behind the biggest changes to the API.

Beyond API changes, this work resulted in the addition of a new
"developer mode". When Babeltrace 2 is built in this mode, a large
number of checks are added to enforce most preconditions required by
the API.

This makes it easier to develop and test new plugins while ensuring
that they honour the contract of the various functions they use. Since
components are now assumed to have been tested in this mode by their
authors, the release configuration of Babeltrace 2 can eschew most
precondition checks at run time, resulting in improved performance.

To build Babeltrace 2 in this mode and test your own plugins, set the
BABELTRACE_DEV_MODE environment variable 1 at configure time, for
example:

BABELTRACE_DEV_MODE=1 ./configure

Moreover, you can use the BABELTRACE_MINIMAL_LOG_LEVEL variable to
compile out all logging below a given level (VERBOSE, DEBUG,
INFO, WARN, ERROR, and FATAL), resulting in further
performance improvements. The performance impact of enabling a logging
level of INFO (and up) is minimal as such statements are seldom used
in hot paths.

The current default logging level is left at VERBOSE and will remain
so until the final release. This setting incurs a performance penalty,
but will make it easier for users of this pre-release to report bugs
with detailed logs.

API changes

The performance improvement work has required significant API changes.
These changes were intrusive and have made it difficult to make
intermediary releases available, which is the main reason for the long
time between releases pre4 and pre5.

The APIs introduced by Babeltrace 2 made use of systematic reference
counting which was identified as a key culprit behind the performance
gap between Babeltrace 1.x and 2.x. A number of APIs routinely used in
the "hot paths" of trace processing now allow references to be
borrowed at no performance cost.

Among the biggest changes is the split between the CTF IR API
(previously introduced by the CTF writer library in Babeltrace 1.x)
and trace IR.

Trace IR is a higher-level trace access API that decouples the trace
processing graph from trace format implementation details. Since it is
decoupled from the CTF specification, we took the opportunity to adopt
a more generic (and sometimes clearer) terminology.

Observations made during the development of the new features of this
release and comments by users who have started using the pre4 API have
led us to adding "self" versions of objects which make use of
compile-time type checking to ensure that using functions that should
only be used in a given context results in warnings when they are
misused.

Moreover, const-related changes to the API will make it easier for
plugin authors to write const-correct code and provide an indication
when certain operations should not be performed.

Finally, an optional message iterator seeking API has been added to
allow seeking to a trace's beginning or to an arbitrary point
expressed in nanoseconds since the trace's origin.

We expect the API to undergo some minor changes before the first
release candidate. Among them figure:

• Addition of ranges associated to variant field classes.
• Introduction of a dependency link between upstream and
  downstream message iterators.
• Introduction of canceler objects, allowing long graph operations to be
  canceled (mainly meant for interactive use cases).
• Introduction of new types and user attributes in preparation for
  CTF 2.
• Versioning API to allow plugins to query graph capabilities.
• Introduction of unsigned/signed value objects.

A number of users had reported having started implementing plugins
using the pre4 API. Hence, we welcome any feedback regarding the API
changes.

Co-installation with Babeltrace 1.x

In order to facilitate the transition from Babeltrace 1.x to 2.x, we
will be making it easier to co-install both release series of
Babeltrace. To do so, the following changes are upcoming planned:

• Rename the CLI tool to babeltrace2. It is planned that
  packagers will symlink it as babeltrace when preferred, as
  is done for CPython packages.
• Rename the include directory to babeltrace2.
• Rename the library to libbabeltrace2.

Since co-installing both series will be possible, we plan to:

• Drop support for the babeltrace Python package (superseded by bt2).
• Drop support for the babeltrace-log util (you can do the
  equivalent with the Babeltrace 2 CLI).

Bug fixes

A large number of bug fixes are introduced as part of this release,
notably:

• Network efficiency improvements of the src.ctf.lttng-live
  component class.
• Fixes to the argument parsing of the CLI.
• CTF writer correctness fixes.
• Miscellaneous bug fixes in the various plugins.

Road to the final release

Here is a non-exhaustive list of changes planned before we ...