Lazy and prefiltered OPTIONAL

[joka921]

So far, OPTIONAL worked with lazy input only for the case of a single join column, and prefiltering was not yet implemented at all. Now OPTIONAL works with lazy input and prefiltering for both of the following two use cases. First, when there is only one join column. Second, when there are two join columns, one of which provably does not contain any UNDEF values. Here is an example query, where the first OPTIONAL is of the first kind (single join column ?astronaut) and the second OPTIONAL is of the second kind (join on ?astronaut and ?label, where only ?label can be UNDEF):

SELECT ?astronaut ?label WHERE {
  ?astronaut wdt:P106 wd:Q11631 .
  OPTIONAL { ?astronaut rdfs:label ?label FILTER (LANG(?label) = "hi") }
  OPTIONAL { ?astronaut rdfs:label ?label FILTER (LANG(?label) = "en") }
}

On Wikidata, the query returns around 1000 astronauts, with their Hindi name if it exists, and their English name otherwise. With the current code, the query takes around 1s (because of the lack of prefiltering). With this change, the query takes around 50ms. This is a dramatic improvement, which is typical when adding labels to a relatively small set of subjects

[codecov]

Codecov Report

❌ Patch coverage is 38.86926% with 346 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.87%. Comparing base (f35a290) to head (237159e).

Files with missing lines	Patch %	Lines
src/index/CompressedRelation.cpp	0.00%	147 Missing ⚠️
src/engine/OptionalJoin.cpp	41.80%	66 Missing and 5 partials ⚠️
src/engine/JoinWithIndexScanHelpers.h	0.00%	44 Missing ⚠️
src/engine/IndexScan.cpp	37.03%	31 Missing and 3 partials ⚠️
src/util/JoinAlgorithms/JoinColumnMapping.h	0.00%	24 Missing ⚠️
src/util/JoinAlgorithms/JoinAlgorithms.h	90.37%	0 Missing and 13 partials ⚠️
src/engine/JoinHelpers.h	74.07%	7 Missing ⚠️
src/engine/AddCombinedRowToTable.h	0.00%	4 Missing ⚠️
src/engine/Operation.h	0.00%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #2695      +/-   ##
==========================================
- Coverage   91.60%   90.87%   -0.74%     
==========================================
  Files         483      484       +1     
  Lines       41386    41890     +504     
  Branches     5496     5552      +56     
==========================================
+ Hits        37912    38067     +155     
- Misses       1897     2220     +323     
- Partials     1577     1603      +26     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[sparql-conformance]

Overview

Number of Tests	Passed ✅	Intended ✅	Failed ❌	Not tested
547	449	73	25	0

Conformance check passed ✅

No test result changes.

Details: https://qlever.dev/sparql-conformance-ui?cur=237159e1311784b204aa1f8b4f2a9f67c3447e6b&prev=f35a290fc35e28fefdc9ac56139660fad14ab860

[sonarqubecloud]

Quality Gate failed

Failed conditions
7.6% Duplication on New Code (required ≤ 3%)
E Reliability Rating on New Code (required ≥ A)

See analysis details on SonarQube Cloud

Catch issues before they fail your Quality Gate with our IDE extension SonarQube for IDE