behaviour of QLever in WDbench opts

[pfps]

WDbench opts is the only benchmark run that I'm currently doing that is at all problematic in QLever, with 71 of the 498 queries failing, almost all due to running out of time when sorting.

Are there plans to handle OPTIONAL better, i.e., to take advantage of the bindings in the rest of the query?

[hannahbast]

@pfps The suboptimal OPTIONAL is fixed by #2673, which will be merged in the next days. We just didn't get to it yet. In particular, please wait with the final version of the Scholia paper until this is fixed. It would be unnecessary negative press otherwise

Here, in a nutshell, why the current implementation of OPTIONAL is suboptimal. When joining a small table with a large table, QLever prefilters the large table, that is, it discards all blocks from the large table that cannot possibly match any row in the small table. These discarded blocks then have to be neither read (from disk) nor decompressed. This hasn't been implemented yet for OPTIONAL (and MINUS and a few other advanced joins) for no good reason except time.

This is particularly relevant for Wikidata queries, where you often have one ore more OPTIONAL { ?subject rdfs:label ?label FILTER LANG(?label) = "..." }, and the left side of this is usually small (few entities for which you want labels), and the right side is large (rdf:label has many triples, even when restricted to only English labels)

[pfps]

I don't think that there is anything in the current Scholia paper on this kind of performance issue. The expanded version isn't likely to be written for a while yet.