Adding custom SPARQL query functions

[hannahbast]

Here is an example how to add a custom SPARQL question function to the QLever codecase. The function squares the given number and can be invoked by an expression like SQR(?x) in a SPARQL query.

1. In https://github.com/ad-freiburg/qlever/blob/master/src/engine/sparqlExpressions/NaryExpression.h, add the following code. This defines a unary function (hence "NARY" and "1") with a numerical argument (hence "NumericValueGetter"), which can then be accessed when evaluating a SPARQL query (using "sparqlExpression::squareExpression", see step 2).

inline auto square = [](auto x) -> double { return x * x; };
using SquareExpression = NARY<1, FV<decltype(square), NumericValueGetter>>;
using detail::SquareExpression;

2. In https://github.com/ad-freiburg/qlever/blob/master/src/parser/sparqlParser/generated/SparqlAutomatic.g4 (the definition of the SPARQL grammar), in the rule for "brackettedExpression", add the following (to the long list of functions that is already there). This will let the parser accept an expression like SQR().

| SQR '(' expression ')'

3. Rebuild the automatically generated ANTLR code using the instructions in https://github.com/ad-freiburg/qlever/blob/master/src/parser/sparqlParser/generated/README.md (in a nutshell: run the ANTLR jar in the respective directory).

4. In https://github.com/ad-freiburg/qlever/blob/master/src/parser/sparqlParser/SparqlQleverVisitor.h, add the following else-if to antlrcpp::Any visitBuiltInCall . This will link the SQR in the SPARQL query to an invocation of the function sparqlExpression::SquareExpression implemented in Step 1.

else if (ad_utility::getLowercase(context->children[0]->getText()) == "sqr") {
  auto child = std::move(visitExpression(context->expression()[0]).as<ExpressionPtr>());
  return createExpression<sparqlExpression::SquareExpression>(std::move(child));
}

[hannahbast]

@AndreaWesterinen Above is an example of how a custom query function can be added to QLever. Johannes (@joka921) implemented the foundation for this three months ago. We are thinking about how this can be simplified further, so that the necessary changes for adding a single function have to be made in only one place.

[LorenzBuehmann]

This is "just" for functions, right? Like using it in FILTER, BIND or projection? Is there also some kind of feature for custom "magic predicates"?

[hannahbast]

@LorenzBuehmann Sorry, I missed this question of yours and just found it by coincidence. Can you give an example of such a "magic predicate"?

[LorenzBuehmann]

no worries @hannahbast - I can see from all the commits and issues that you're quite busy with QLever (besides other projects and teaching stuff of course). I see that I should have explained the term "magic predicate" which obviously isn't a standard at all ...

What I mean is that particular predicates are handled/evaluated differently than the domain predicates. This is a quite common approach, for example to make use of a text index or a geospatial index or anything else. Technically, from the implementations I have seen so far, a triple pattern with such a predicate would indeed also produce bindings. And indeed, quite often we could still stick to your custom functions and use those in a BIND - but not if the custom service would generate multiple solutions as BIND doesn't generate a multiset of bindings afaik.

For example,

SELECT * WHERE {
?s a :Place .
?s text:query "(label:foo 10)" .
}

could make use of e.g. a Lucene fulltext index to get 10 bindings for ?s matching the term "foo" of field label. It's just a sketch, but that's how e.g. Jena, Virtuoso, GraphDB and others integrate fulltext search.

But I can see that depending on the used database paradigm, it might be more or less complicated to support such kind of hook in the code base. Nevertheless, you already make use of those predicates, e.g. ql:contains-word - the question would be how difficult it would be to bring in other predicates.

Thanks for support.