RFC-0008: Tracebox Explicit Daemon Management

[github-actions[bot]]

📄 RFC Doc: 0008-tracebox-explicit-daemon-management.md

---

Tracebox Explicit Daemon Management

Authors: @sashwinbalaji, @primiano

Status: Draft

Problem

tracebox is a key tool for using Perfetto, especially on Linux and for
development purposes, bundling traced, traced_probes, and the perfetto CLI
into a single binary. However, its current "autostart" mode—where invoking
tracebox with trace arguments automatically spawns daemons—has several
structural issues that create a confusing and unreliable user experience.

1. Daemon Lifecycle and SDK Integration: In its default autostart mode,
   tracebox spawns traced and traced_probes only for the duration of the
   command's execution. This ephemeral nature, coupled with the use of private,
   temporary sockets, creates significant problems:

   • Perfetto SDK Users: Processes instrumented with the Perfetto SDK
     (e.g., using track_event) expect a persistent tracing service on
     standard system sockets. When an SDK-instrumented application starts, it
     fails to connect because no daemon is running. If tracebox is then
     used to trace, it spawns its own daemons on private sockets (e.g.,
     PID-based abstract sockets on Linux) which the already-running
     application cannot discover. This results in traces missing all SDK data
     and is a major source of developer confusion, as highlighted in issues
     like #3437,
     #2105, and
     #850.
   • Other Producers: External tools (e.g., Mesa) that act as trace
     producers also require a stable, accessible tracing service and face the
     same discovery issues.

2. User Experience: It is not obvious to users whether the required daemons
   are running or which sockets are in use. The lack of a persistent service
   model on non-Android platforms is contrary to developer expectations for
   system tracing tools.

3. System Integration: tracebox lacks a straightforward way to set up a
   persistent, system-wide tracing service. For users on systemd-based
   distributions without a Debian package, there is no simple migration path
   from a temporary setup to a proper service installation. Furthermore, the
   current autostart behavior would conflict with or hide daemons managed by a
   future apt install of Perfetto.

4. Platform Inconsistency: The daemon management behavior is inconsistent
   across platforms. On Linux/Android, it uses PID-based abstract domain
   sockets (@traced-c-PID) that auto-cleanup, while on macOS it creates
   filesystem sockets (/tmp/traced-c-PID) that are left stale after a crash.
   On Windows, the autostart mode is entirely unsupported and triggers a
   PERFETTO_FATAL, making any cross-platform workflow unreliable.

5. Session Clashes: Running multiple tracebox instances simultaneously
   leads to conflicts as they compete for control of system resources like
   ftrace and the same socket paths. This can result in catastrophic failures
   and undefined behavior, as noted in
   #2903.

Decision

To provide a clearer, more robust, and less mysterious model, tracebox's
daemon management will become explicit. This is a breaking behavioral change.

1. Explicit Daemon Control by Default: Running tracebox with trace
   configuration arguments (e.g., tracebox -c config.pbtx) will no longer
   automatically start traced and traced_probes. It will require the
   daemons to be already running and accessible on standard system sockets
   (e.g., /run/perfetto/... or /tmp/perfetto/...).

2. New tracebox ctl Applet for Daemon Management: A new applet is
   introduced to manage the lifecycle of daemons for users not relying on a
   system package manager. It supports two distinct modes of operation:

   • User-Session Daemons:
     • tracebox ctl start: Starts traced and traced_probes as
       detached background daemons for the current user, using sockets
       in /tmp/. This is for temporary, non-system-wide use.
     • tracebox ctl stop: Stops the daemons started via ctl start.
     • tracebox ctl status: Reports the status of these daemons.
   • System-Wide Service Installation:
     • tracebox ctl install-systemd-units [--start]: For users on
       systemd-based systems, this command generates and installs
       systemd unit files, effectively replicating a system package
       installation. This is a one-time setup action for a persistent,
       system-wide service.

3. Backward Compatibility via Flag: The current self-contained execution
   model is preserved for existing scripts via the --autodaemonize=session
   flag.

4. Clear Guidance: If tracebox is invoked and daemons are not detected,
   it will exit with an actionable error message, pointing to systemctl (for
   system-managed installs) or tracebox ctl start (for user-managed daemons).

Design

tracebox Default Mode Changes

When tracebox is invoked without a specific applet name (e.g., tracebox -c config.pbtx ...):

• --autodaemonize=session: This flag preserves the current behavior.
  Daemons are spawned on private, temporary sockets (PID-based abstract
  sockets on Linux/Android, /tmp/traced-*-PID on macOS) and live only for
  the duration of the command.
• --autodaemonize=none (or flag omitted): This is the new default
  behavior.
  1. Daemon Check: It attempts a non-blocking connection to the expected
     socket locations to find active daemons. The search order is:
     • Environment Variables: PERFETTO_*_SOCK_NAME
     • Systemd path: /run/perfetto/traced-producer.sock
     • User-space path: /tmp/perfetto-producer.sock
  2. On Success: It uses the first responsive sockets found and proceeds
     with the trace.
  3. On Failure: It exits with an error code and prints a clear
     message:

     Error: Perfetto tracing daemons (traced, traced_probes) are not running.
     - If using a system package (e.g., Debian): `sudo systemctl start perfetto.svc`
     - To run daemons as the current user: `tracebox ctl start`
     - For a self-contained run: `tracebox --autodaemonize=session ...`
     See [link to docs] for more information.
     

tracebox ctl Applet

This applet provides two distinct daemon management workflows:

• User-Session Management:

  • tracebox ctl start:
  • Refuses to run if a systemd service for Perfetto is detected (e.g., by
    checking for /lib/systemd/system/perfetto.svc), guiding the user to
    systemctl.
  • Checks if daemons are already running on the user-space sockets
    (/tmp/perfetto-*.sock). If so, it reports this and exits.
  • Spawns traced and traced_probes as detached processes owned by the
    current user, using standard daemonization techniques (e.g., double
    fork).
  • Uses socket paths in /tmp/ (e.g.,
    /tmp/perfetto-{producer,consumer}.sock) with permissions for the
    current user.
  • Stores PID files (e.g., in /tmp/) for management.
  • tracebox ctl stop: Finds the daemons via their PID files and sends
    a termination signal.
  • tracebox ctl status: Checks if the user-session daemons are
    responsive on the /tmp/ sockets.

• System-Wide Service Installation:

  • tracebox ctl install-systemd-units: This command (requiring
    sudo) generates traced.service, traced_probes.service, and a
    perfetto.svc target and installs them into /etc/systemd/system/.
    This provides a persistent setup that survives reboots.
  • The optional --start flag will also execute systemctl daemon-reload
    and systemctl start perfetto.svc.
  • This command should refuse to run if package-managed unit files are
    already detected (e.g., in /lib/systemd/system/).

Interaction with Debian Package

• A Debian package will install systemd units to manage daemons using
  sockets in /run/perfetto/. This is the canonical method.
• The perfetto and tracebox binaries will prioritize connecting to the
  /run/perfetto/ sockets.
• The tracebox ctl start and tracebox ctl install-systemd-units commands
  will yield to a Debian package installation to prevent conflicts.

Socket Paths Summary

Management Method	Producer/Consumer Socket Paths	Persistence
Systemd (Debian)	/run/perfetto/{producer,consumer}.sock	Persistent
tracebox ctl	/tmp/perfetto-{producer,consumer}.sock	Persistent
--autodaemonize=session	Abstract sockets or /tmp/traced-{c,p}-PID	Ephemeral

SDK Enhancements

• While out of scope for this RFC, the client library's reconnection logic
  could be enhanced to use mechanisms like inotify on Linux to detect socket
  creation and trigger an immediate reconnection attempt.
  (WIP CL)

Alternatives Considered

1. Keep Current Behavior: Rejected. Fails to solve critical SDK integration
   problems.
2. Daemonize Forever Automatically: Rejected as explicit control is more
   cleaner.
3. Time-Limited Daemons (e.g., 1-hour or 24-hour): Rejected. This approach
   was deemed too complex and "mysterious". Reliably managing timeouts,
   handling edge cases with active tracing sessions, and debugging issues
   across platforms would introduce significant maintenance overhead. The
   explicit-control model is simpler and more predictable.

Open Questions

• Do we really need install-systemd-units option as if Debian packaging goes
  well that should cover most of the cases and mixing both can lead to
  confusion.
• What is the appropriate mechanism for tracebox ctl commands to detect a
  system-wide Perfetto installation to yield control to systemctl?
• Detailed implementation of daemonization and PID management for ctl stop
  on all supported platforms (Linux, Mac, Windows).

---

💬 Discussion Guidelines:

• This discussion is automatically synced with the RFC document
• Please provide constructive feedback and suggestions

[sashwinbalaji]

fyi: @ribalda

[ribalda]

I agree with the content of the RFC, some minor comments:

nit: why tracebox ctl and not tracebox systemd ? I think systemd is more explicit. (unless you think that you will have to support other OSs or /sbin/init)

tracebox ctl start:
Refuses to run if a systemd service for Perfetto is detected (e.g., by
checking for /lib/systemd/system/perfetto.svc), guiding the user to
systemctl.

It should refuse to run if a service is RUNNING, not if it is installed. Most of the times, distros will provide their own .service files following their policies. Some distros will prefer to leave the service running from boot, other distros will prefer that users/root start/stop the service.

Interaction with Debian Package
A Debian package will install systemd units to manage daemons using
sockets in /run/perfetto/. This is the canonical method.
The perfetto and tracebox binaries will prioritize connecting to the
/run/perfetto/ sockets.
The tracebox ctl start and tracebox ctl install-systemd-units commands
will yield to a Debian package installation to prevent conflicts.

I do not think that there is needed any special behavior with Debian. ctl install-systemd-units should ask systemd if there is a service installed and yield if true, regardless of the distro used.

[LalitMaganti]

Two thoughts from me:

1. Please let's properly publcise this change with good quality error messages and documentation and release notes. There's nothing worse than a tool breaking and not understanding why
2. What are we going to do about --system-sockets? Are we going to keep backwards compatibility of this or not?

[primiano]

Thanks a lot for putting this together!!! 😮
LGTM with very minimal comments.

A few notes:

perfetto.svc

I don't think we need this systemd unit. What does this do? perfetto is only a cmdline client, not a service. Unless you want to use this as a "group" somehow to start/stop both? (clarify if it's the case)

nit: why tracebox ctl and not tracebox systemd ? I think systemd is more explicit. (unless you think that you will have to support other OSs or /sbin/init)

I think "ctl" is right. what he plans to do (I think) is to NOT use systemd when doing ctl start.
This is deliberately for use in systems that don't have systemd, where perfetto has NOT been installed via the package manager.

it's some sort of "apachectl"

Realistically this will just daemonize() the two services and remember their pid somewhere.

We need to have a way to cheaply run the daemon without having to install anything.
If I want to try perfetto I'd like to do so without having to install any package.

@sashwinbalaji I have one caveat to add to the way "ctl" works. IMHO it:

1. detect if we are in a debian installation.(this can be plumbed through a #define / GN var in the debian package)
2. detect if there is a systemd unit installation
   if 1 or 2 are true, it should invoke "systemd start xxx" and print a message along the lines of "I see you have a systemd unit for this, let's use that, rather than doing our own daemon ctl"

If !1 && !2 instead it should do its own daemonization and store the pids in /tmp/somewhere (or just pkill to stop?)

[ribalda]

I think "ctl" is right. what he plans to do (I think) is to NOT use systemd when doing ctl start.
This is deliberately for use in systems that don't have systemd, where perfetto has NOT been installed via the package manager.

ACK, I agree thanks for the explanation

1. detect if we are in a debian installation.(this can be plumbed through a #define / GN var in the debian package)

I think it should be irrelevant if this is a Debian installation or not. The only thing that matters is if there is a systemd unit.

[primiano]

Ah another comment.

tracebox ctl | /tmp/perfetto-{producer,consumer}.sock

I think in tracebox ctl we should try to use /run/perfetto/ if

1. we are root
2. the user has happened to chmod properly /run/perfetto to $USER

(we don't need to probe for root, we should just try to access or mkdir that dir)

The reason is the SDK. If people want to try out the sdk, it should JustWork(TM) if they start tracebox ctl as root

If we fail to acquire the default socket path, we should print a very explicit error message that says that we are using a fallback path, and the SDK will NOT work (Which is what catches everybody by surprise)