Concurrency in columnlineage get #57
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This pull request refactors the lineage fetching logic in the
ColumnLineageService
and updates related tests. The changes focus on improving concurrency for lineage data retrieval, simplifying the API by replacing thelineage
method withdirectColumnLineage
, and aligning test cases with the new implementation.Refactoring lineage fetching logic:
ColumnLineageService.java
: Replaced thelineage
method withdirectColumnLineage
, introducing concurrency usingCompletableFuture
to fetch upstream and downstream lineage simultaneously whenwithDownstream
is true. Added error handling for concurrent execution failures.API updates:
ColumnLineageResource.java
: Updated thegetLineage
method to use the newdirectColumnLineage
method instead of the removedlineage
method.Test updates:
ColumnLineageResourceTest.java
: Adjusted mock expectations to calldirectColumnLineage
instead oflineage
.ColumnLineageServiceTest.java
: Updated all test cases to align with the newdirectColumnLineage
method, removed assertions on graph contents since the new implementation returns empty graphs when no lineage data is found, and added tests for concurrent execution and varying depth values. [1] [2] [3] [4] [5] [6] [7] [8]Code cleanup:
ColumnLineageServiceTest.java
, simplifying the test file. [1] [2]Checklist