Skip to content

Comments

Do not allow setting TableConfig in IndexLoadingConfig after construction#14098

Merged
Jackie-Jiang merged 1 commit intoapache:masterfrom
Jackie-Jiang:index_loading_config
Sep 27, 2024
Merged

Do not allow setting TableConfig in IndexLoadingConfig after construction#14098
Jackie-Jiang merged 1 commit intoapache:masterfrom
Jackie-Jiang:index_loading_config

Conversation

@Jackie-Jiang
Copy link
Contributor

We should not allow setting TableConfig into IndexLoadingConfig after it is constructed because the step of extracting index settings are not performed.
Ideally, we should not allow any setter in IndexLoadingConfig, and it should always honor table config and schema. That involves bigger change, and will be addressed in followup PRs

@codecov-commenter
Copy link

codecov-commenter commented Sep 27, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 64.03%. Comparing base (59551e4) to head (5ab8b0d).
Report is 1097 commits behind head on master.

Additional details and impacted files
@@             Coverage Diff              @@
##             master   #14098      +/-   ##
============================================
+ Coverage     61.75%   64.03%   +2.28%     
- Complexity      207     1537    +1330     
============================================
  Files          2436     2596     +160     
  Lines        133233   143341   +10108     
  Branches      20636    21956    +1320     
============================================
+ Hits          82274    91787    +9513     
+ Misses        44911    44800     -111     
- Partials       6048     6754     +706     
Flag Coverage Δ
custom-integration1 100.00% <ø> (+99.99%) ⬆️
integration 100.00% <ø> (+99.99%) ⬆️
integration1 100.00% <ø> (+99.99%) ⬆️
integration2 0.00% <ø> (ø)
java-11 63.98% <ø> (+2.27%) ⬆️
java-21 63.90% <ø> (+2.28%) ⬆️
skip-bytebuffers-false 64.00% <ø> (+2.25%) ⬆️
skip-bytebuffers-true 63.87% <ø> (+36.14%) ⬆️
temurin 64.03% <ø> (+2.28%) ⬆️
unittests 64.02% <ø> (+2.28%) ⬆️
unittests1 55.71% <ø> (+8.82%) ⬆️
unittests2 34.46% <ø> (+6.73%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

}

@VisibleForTesting
public void setTableConfig(TableConfig tableConfig) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

I believe logic based on _dirty = true; would be fine, as there are a few other methods setting this flag anyway.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The main problem is that the setters are not reflected back to table config, which is causing inconsistency across configs

}

/**
* NOTE: Can be used in production code when we want to load a segment as is without any modifications.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would this conflict with the case when user indeed wants to 'remove' all indexes?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When table config and schema are not set, pre-processor is skipped

@Jackie-Jiang Jackie-Jiang merged commit fce17d3 into apache:master Sep 27, 2024
@Jackie-Jiang Jackie-Jiang deleted the index_loading_config branch October 2, 2024 21:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants