Skip to content

Feature Request: Option to Store Raw Kafka Messages in S3 Without Parser Transformation in Secor. #2826

@Vijay1518

Description

@Vijay1518

Problem: Secor’s current parsers require serialization and deserialization of Kafka messages before storing them in S3, which alters the original message format. For cases where we need to store raw, unaltered messages, the existing parsers (Json parser )add unnecessary transformations.

Expected Behavior: Introduce an option to bypass parser transformations and save messages in S3 exactly as they are received from Kafka, without serialization, deserialization, or any modification.

how can i fix this?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions