Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Get PrefilterExpression from SparqlExpression #1613

Merged

Conversation

realHannes
Copy link
Collaborator

@realHannes realHannes commented Nov 12, 2024

The SparqlExpression base class has been extended with the method getPrefilterExpressionForMetadata. This method constructs for suitable (logical) expressions which are used inside a FILTER a corresponding PrefilterExpression (see PR #1503). These PrefilterExpressions can be used to prefilter the blocks of an IndexScan by only looking at their metadata.
At the moment, the following expressions provide an overriden implementation of getPrefilterExpressionForMetadata: strstarts (preliminary), logical-or and logical-and (binary), logical-not (unary) and the standard RelationalExpressions (<, ==, >, <=, >=).

realHannes and others added 30 commits April 25, 2024 19:07
Copy link

codecov bot commented Nov 12, 2024

Codecov Report

Attention: Patch coverage is 90.61372% with 26 lines in your changes missing coverage. Please review.

Project coverage is 89.25%. Comparing base (6fba76f) to head (0280d90).
Report is 1 commits behind head on master.

Files with missing lines Patch % Lines
...ine/sparqlExpressions/PrefilterExpressionIndex.cpp 88.07% 3 Missing and 10 partials ⚠️
...engine/sparqlExpressions/RelationalExpressions.cpp 80.43% 1 Missing and 8 partials ⚠️
...engine/sparqlExpressions/SparqlExpressionPimpl.cpp 0.00% 3 Missing ⚠️
...ine/sparqlExpressions/NumericBinaryExpressions.cpp 98.30% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1613      +/-   ##
==========================================
- Coverage   89.25%   89.25%   -0.01%     
==========================================
  Files         372      372              
  Lines       34818    35089     +271     
  Branches     3931     3971      +40     
==========================================
+ Hits        31076    31317     +241     
- Misses       2470     2482      +12     
- Partials     1272     1290      +18     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Member

@joka921 joka921 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A thorought round of reviews again,
I'll have a look at the linking issues, the rest should be self-explanatory.

src/index/CompressedBlockPrefiltering.h Outdated Show resolved Hide resolved
src/index/CompressedBlockPrefiltering.h Outdated Show resolved Hide resolved
src/index/CompressedBlockPrefiltering.h Show resolved Hide resolved
src/index/CompressedBlockPrefiltering.h Outdated Show resolved Hide resolved
src/index/CompressedBlockPrefiltering.cpp Show resolved Hide resolved
src/engine/sparqlExpressions/SparqlExpressionPimpl.cpp Outdated Show resolved Hide resolved
src/engine/sparqlExpressions/SparqlExpression.h Outdated Show resolved Hide resolved
Comment on lines 1077 to 1103
//______________________________________________________________________________
// For this test we expect that no PrefilterExpression is available.
TEST(SparqlExpression, getEmptyPrefilterFromSparqlRelational) {
const Variable var = Variable{"?x"};
const Iri iri = I("<Iri>");
const Literal lit = L("\"lit\"");
evalAndEqualityCheck(leSprql(var, var));
evalAndEqualityCheck(neqSprql(iri, var));
evalAndEqualityCheck(eqSprql(var, iri));
evalAndEqualityCheck(neqSprql(IntId(10), DoubleId(23.3)));
evalAndEqualityCheck(gtSprql(DoubleId(10), lit));
evalAndEqualityCheck(ltSprql(VocabId(10), BoolId(10)));
evalAndEqualityCheck(geSprql(lit, lit));
evalAndEqualityCheck(eqSprql(iri, iri));
evalAndEqualityCheck(orSprqlExpr(eqSprql(var, var), gtSprql(var, IntId(0))));
evalAndEqualityCheck(orSprqlExpr(eqSprql(var, var), gtSprql(var, var)));
evalAndEqualityCheck(andSprqlExpr(eqSprql(var, var), gtSprql(var, var)));
}

//______________________________________________________________________________
// For the following more complex SparqlExpression trees, we also expect an
// empty PrefilterExpression vector.
TEST(SparqlExpression, getEmptyPrefilterForMoreComplexSparqlExpressions) {
const Variable varX = Variable{"?x"};
const Variable varY = Variable{"?y"};
const Variable varZ = Variable{"?z"};
// ?x <= 10.00 OR ?y > 10
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you please create a separate ...Test.cpp file for the tests that test the creation of the prefilterThings from SparqlExpressions.
2. Can you give the test suite a unique name that refers to the filename? It is hard to find tests when there is 4 different files where we have TEST(SparqlExpression, ...) in them.

So overall there's (at least) three files:

  1. The tests for the SparqlExpressions (already present).
  2. The file for the Prefiltering without the expressions (this file before the PR + the clone and format and equal tests)
  3. The (new) file, that tests the combination of SparqlExpressions with the prefilterExpressions (test cases already there, so this shouldn't be too much work).

@sparql-conformance
Copy link

Copy link

sonarcloud bot commented Nov 13, 2024

Quality Gate Passed Quality Gate passed

Issues
1 New issue
2 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

Copy link
Member

@joka921 joka921 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you very much.

@joka921 joka921 merged commit 155718d into ad-freiburg:master Nov 14, 2024
22 checks passed
realHannes added a commit to realHannes/qlever that referenced this pull request Nov 15, 2024
The `SparqlExpression` base class has been extended with the method `getPrefilterExpressionForMetadata`. This method constructs for suitable (logical) expressions which are used inside a `FILTER` a corresponding `PrefilterExpression` (see PR ad-freiburg#1503). These `PrefilterExpression`s can be used to prefilter the blocks of an `IndexScan` by only looking at their metadata.
At the moment, the following expressions provide an overriden implementation of `getPrefilterExpressionForMetadata`: `strstarts` (preliminary), `logical-or` and `logical-and` (binary),  `logical-not` (unary) and the standard `RelationalExpressions (<, ==, >, <=, >=)`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants