Skip to content

Concurrency in columnlineage get #57

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jun 17, 2025

Conversation

jonathanpmoraes
Copy link

This pull request refactors the lineage fetching logic in the ColumnLineageService and updates related tests. The changes focus on improving concurrency for lineage data retrieval, simplifying the API by replacing the lineage method with directColumnLineage, and aligning test cases with the new implementation.

Refactoring lineage fetching logic:

  • ColumnLineageService.java: Replaced the lineage method with directColumnLineage, introducing concurrency using CompletableFuture to fetch upstream and downstream lineage simultaneously when withDownstream is true. Added error handling for concurrent execution failures.

API updates:

  • ColumnLineageResource.java: Updated the getLineage method to use the new directColumnLineage method instead of the removed lineage method.

Test updates:

  • ColumnLineageResourceTest.java: Adjusted mock expectations to call directColumnLineage instead of lineage.
  • ColumnLineageServiceTest.java: Updated all test cases to align with the new directColumnLineage method, removed assertions on graph contents since the new implementation returns empty graphs when no lineage data is found, and added tests for concurrent execution and varying depth values. [1] [2] [3] [4] [5] [6] [7] [8]

Code cleanup:

  • Removed unused imports and redundant code in ColumnLineageServiceTest.java, simplifying the test file. [1] [2]

Checklist

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • New and existing unit tests pass locally with my changes

@jonathanpmoraes jonathanpmoraes merged commit d5a7915 into main Jun 17, 2025
4 checks passed
@jonathanpmoraes jonathanpmoraes deleted the concurrency-in-columnlineage-get branch June 17, 2025 15:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants