Skip to content

option to ignore missing fields at the end of a log #62

@mmguero

Description

@mmguero

This is an enhancement request.

The documentation says:

When dissecting a string from left to right, text is captured upto the first delimiter - this captured text is stored in the first field. This is repeated for each field/# delimiter pair thereafter until the last delimiter is reached, then the remaining text is stored in the last field.

However, when you have the opposite problem (eg., if you have missing fields instead of the correct number of fields or missing fields) you get a "Dissector mapping, pattern not found" error.

This makes it difficult to use one set of rules to parse logs with a format that evolves over time, adding new fields to the end.

It would be very convenient if there were an option to just set the missing fields to nil, or not add them to the event at all.

Activity

guyboertje

guyboertje commented on Jul 31, 2018

@guyboertje

We will be adding this [1] fairly soon. Its coded up but needs some tidying.
[1] #56
You may find it will help to define a number of patterns that will get tried until success.

mmguero

mmguero commented on Aug 22, 2018

@mmguero
Author

I see, thanks for the comment. Yes, I think that would allow me to do what I'm attempting. I'll look forward to the next release.

mmguero

mmguero commented on Dec 20, 2018

@mmguero
Author

Just curious, do you have any thoughts on when #56 will be ready for release? Just trying to gauge what "fairly soon" means for whether or not I'll be able to use that enhancement in a project I'm working on.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

    Development

    No branches or pull requests

      Participants

      @guyboertje@mmguero

      Issue actions

        option to ignore missing fields at the end of a log · Issue #62 · logstash-plugins/logstash-filter-dissect