Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve dimension table handing #13967

Merged
merged 1 commit into from
Sep 10, 2024

Conversation

Jackie-Jiang
Copy link
Contributor

  • Introduce finer control and access on dimension table
  • Do not always read the whole row for each lookup. In most cases we only need to access one single value

@@ -219,8 +218,7 @@ private void lookup(ValueBlock valueBlock, ValueAcceptor valueAcceptor) {
}
}
// lookup
GenericRow row = _dataManager.lookupRowByPrimaryKey(primaryKey);
Object value = row == null ? null : row.getValue(_dimColumnName);
Object value = _dataManager.lookupValue(primaryKey, _dimColumnName);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice

but where lookupValues() method is used? or added for future use

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sharp eyes! Yes it is a preparation for #13966

@codecov-commenter
Copy link

codecov-commenter commented Sep 9, 2024

Codecov Report

Attention: Patch coverage is 68.75000% with 10 lines in your changes missing coverage. Please review.

Project coverage is 57.88%. Comparing base (59551e4) to head (6630e8b).
Report is 1017 commits behind head on master.

Files with missing lines Patch % Lines
...manager/offline/MemoryOptimizedDimensionTable.java 23.07% 9 Missing and 1 partial ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##             master   #13967      +/-   ##
============================================
- Coverage     61.75%   57.88%   -3.87%     
- Complexity      207      219      +12     
============================================
  Files          2436     2612     +176     
  Lines        133233   143197    +9964     
  Branches      20636    21986    +1350     
============================================
+ Hits          82274    82887     +613     
- Misses        44911    53832    +8921     
- Partials       6048     6478     +430     
Flag Coverage Δ
custom-integration1 <0.01% <0.00%> (-0.01%) ⬇️
integration <0.01% <0.00%> (-0.01%) ⬇️
integration1 <0.01% <0.00%> (-0.01%) ⬇️
integration2 0.00% <0.00%> (ø)
java-11 57.86% <68.75%> (-3.85%) ⬇️
java-21 57.77% <68.75%> (-3.86%) ⬇️
skip-bytebuffers-false 57.88% <68.75%> (-3.87%) ⬇️
skip-bytebuffers-true 57.75% <68.75%> (+30.02%) ⬆️
temurin 57.88% <68.75%> (-3.87%) ⬇️
unittests 57.87% <68.75%> (-3.87%) ⬇️
unittests1 40.77% <68.75%> (-6.12%) ⬇️
unittests2 27.91% <0.00%> (+0.18%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@Jackie-Jiang Jackie-Jiang merged commit d04e361 into apache:master Sep 10, 2024
20 of 21 checks passed
@Jackie-Jiang Jackie-Jiang deleted the enhance_dimension_table branch September 12, 2024 18:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants