JSON Array splitting #77
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
51-code
added
assistance
51-code
commented
Oct 1, 2024
51-code
added
review
and removed
assistance
eemhu
reviewed
Oct 2, 2024
eemhu
reviewed
Oct 2, 2024
eemhu
reviewed
Oct 2, 2024
src/main/java/com/teragrep/lsh_01/conversion/DefaultPayload.java
Outdated
Show resolved
Hide resolved
eemhu
reviewed
Oct 2, 2024
src/main/java/com/teragrep/lsh_01/conversion/RegexConversion.java
Outdated
Show resolved
Hide resolved
|
@51-code please rebase |
|
I'll be adding fixes to the config issues today. |
|
Fixed the Config problems. Refactoring of all configuration in the component should be done, which is a bit out of scope for this PR. I'll create a new issue about that. Stuff left out for now is for example default configurations with a decorator object and reworking the config file / system property mix that it now has. Now the end-to-end tests are still using System properties, so I left that possibility in the new ConversionFactory as well. |
eemhu
reviewed
Oct 8, 2024
eemhu
reviewed
Oct 8, 2024
Delete PayloadConfig and include configuration in ConversionFactory that provides the correct object instead Add secondary ctor to RegexConversion Use Collections.singletonList in DefaultPayload, RegexConversion as final class Add object equality test for RegexPayload Move tests to the correct package Move config tests from RegexPayloadTest to PayloadConfigTest and add tests for splitType Refactor split() function in Payload Refactor JsonPayload and RegexPayload to encapsulate Payload instead of String Rename Payload's take() function to message() Implement json_array splitting Refactor Payload objects and regex splitting Remove duplicate tests from RegexSplittingTest Apply spotless Move regex splitting tests to their own file, refactor the tests Change config to allow splitting with json_array Refactor regex splitting to a decorator
|
Rebased. |
eemhu
reviewed
Oct 9, 2024
eemhu
reviewed
Oct 9, 2024
kortemik
requested changes
Oct 9, 2024
src/main/java/com/teragrep/lsh_01/authentication/BasicAuthentication.java
Outdated
Show resolved
Hide resolved
kortemik
requested changes
Oct 14, 2024
kortemik
requested changes
Oct 16, 2024
src/test/java/com/teragrep/lsh_01/MultithreadingEndToEndTest.java
Outdated
Show resolved
Hide resolved
…ed value in assertions
kortemik
approved these changes
Oct 16, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Adds a new splitting option: json_array. Takes HTTP requests with this kind of format:
and converts each of the JSON objects to their own syslog message and sends them forward.
Configuration had changes:
payload.splitEnabledwas changed topayload.splitType. Valid values are regex, json_array or none. none is the default.payload.splitRegexwas moved topayload.splitType.regex.pattern.Regex splitting was already supported, but the design was not scalable. Therefore it was refactored as well to allow json_array to be implemented.
There was a problem with Jakarta's way of getting a JSON object as a string. It removes any whitespace from the String representation. This is okay as the content stays the same.
Tests most importantly include:
JsonPayloadTestfor unit tests for the new functionalityJsonSplittingTestfor end-to-end testing the functionality with actual HTTP-requests sent to the component