tracing-dedup

A Rust crate that deduplicates consecutive identical log events in tracing, showing a repeat count instead of spamming your logs.

What it does

When your application emits the same log message multiple times in a row, tracing-dedup will:

1. Show the first occurrence
2. Suppress subsequent identical messages
3. Display "previous message repeated X times" when a different message appears

This keeps your logs clean and readable while preserving information about how many times each event occurred.

Example

use tracing_subscriber::fmt;
use tracing_dedup::DeduplicatingFormatter;

fn main() {
    tracing_subscriber::fmt()
        .event_format(DeduplicatingFormatter::new(fmt::format::Format::default()))
        .init();

    tracing::info!("First message");
    tracing::info!("First message");
    tracing::info!("First message");
    tracing::info!("Different message");
}

Output:

2025-10-13T17:50:28.654623Z  INFO simple: First message
previous message repeated 3 times
2025-10-13T17:50:28.654652Z  INFO simple: Different message

Features

• Zero overhead when no duplicates: Only tracks state when consecutive duplicates occur
• Respects output destination: Works with stdout, files, or any custom writer
• Level-aware: Messages with different log levels (INFO vs WARN) are treated as distinct
• Target-aware: Same message from different modules/targets are treated as distinct
• Thread-safe: Uses proper locking for concurrent access

Installation

Add this to your Cargo.toml:

[dependencies]
tracing-dedup = "0.1"
tracing = "0.1"
tracing-subscriber = "0.3"

Usage

Basic usage (stdout)

use tracing_subscriber::fmt;
use tracing_dedup::DeduplicatingFormatter;

tracing_subscriber::fmt()
    .event_format(DeduplicatingFormatter::new(fmt::format::Format::default()))
    .init();

With custom writer (file output)

use std::fs::File;
use tracing_subscriber::fmt;
use tracing_dedup::DeduplicatingFormatter;

let log_file = File::create("app.log")?;

tracing_subscriber::fmt()
    .event_format(DeduplicatingFormatter::new(fmt::format::Format::default()))
    .with_writer(move || log_file.try_clone().unwrap())
    .init();

With custom formatting

You can wrap any formatter that implements FormatEvent:

use tracing_subscriber::fmt::format;
use tracing_dedup::DeduplicatingFormatter;

let formatter = format::Format::default()
    .with_level(true)
    .with_target(false)
    .compact();

tracing_subscriber::fmt()
    .event_format(DeduplicatingFormatter::new(formatter))
    .init();

How it works

tracing-dedup implements the FormatEvent trait, wrapping an inner formatter. It:

1. Extracts the message, level, and target from each event
2. Compares with the previous event
3. If identical: increments a counter and skips formatting
4. If different: outputs the repeat count (if any) then formats the new event

This approach ensures that repeat count messages are written to the same destination as your logs (stdout, file, etc.) without triggering recursive logging.

Examples

Run the included examples:

# Basic example
cargo run --example simple

# File output example
cargo run --example custom_writer
cat output.log

License

MIT OR Apache-2.0