Algorithms and Analysis for the SPARQL Constructs
Abstract
As Resource Description Framework (RDF) is becoming a popular data modelling standard, the challenges of efficient processing of Basic Graph Pattern (BGP) SPARQL queries (a.k.a. SQL innerjoins) have been a focus of the research community over the past several years. In our recently published work we brought community’s attention to another equally important component of SPARQL, i.e., OPTIONAL pattern queries (a.k.a. SQL leftouterjoins). We proposed novel optimization techniques – first of a kind – and showed experimentally that our techniques perform better for the lowselectivity queries, and give at par performance for the highly selective queries, compared to the stateoftheart methods.
BGPs and OPTIONALs (BGPOPT) make the basic building blocks of the SPARQL query language. Thus, in this paper, treating our BGPOPT query optimization techniques as the primitives, we extend them to handle other broader components of SPARQL such as such as UNION, FILTER, and DISTINCT. We mainly focus on the procedural (algorithmic) aspects of these extensions. We also make several important observations about the structural aspects of complex SPARQL queries with any intermix of these clauses, and relax some of the constraints regarding the cyclic properties of the queries proposed earlier. We do so without affecting the correctness of the results, thus providing more flexibility in using the BGPOPT optimization techniques.
∎
1 Introduction
Resource Description Framework (RDF) rdf () is being used as the standard for representing semantically linked data on the web as well as for other domains such as biological networks, e.g. UniProt RDF network by Swiss Institute of Bioinformatics^{1}^{1}1http://www.uniprot.org/format/uniprot_rdf. RDF is a directed edgelabeled multigraph, where each unique edge (S P O) is called a triple – P is the label on the edge from the node S to node O, and SPARQL sparql () is the standard query language for it.
SPARQL provides various syntactic constructs to form structured queries over RDF graphs. These constructs have a close similarity to their SQL counterparts. For instance, Basic Graph Patterns (BGP) of SPARQL (or TriplesBlock as referred to in the SPARQL grammar) are similar to the SQL innerjoins (). OPTIONAL patterns of SPARQL (OPTIONALGraphPattern in the SPARQL grammar) are similar to the leftouterjoins (). FILTERs of SPARQL makes up for the SQL LIKE clause and various other selection conditions. UNIONs () and DISTINCTs of SPARQL are similar to their SQL counterparts. GroupGraphPattern of the SPARQL grammar consists of BGP, OPTIONAL, UNION, and FILTER components, and like SQL, SPARQL grammar too allows nested queries with a complex intermix of these query constructs. Since there is an equivalence between SPARQL and SQL constructs, we will use these terms interchangeably in the rest of the text.
BGP queries make the building blocks of SPARQL, and just like SQL innerjoins, they are associative and commutative, i.e., a change in the order of joins among the triple patterns does not change the final results – thus allowing a reorderability among the BGP triple patterns. Owing to these similarities, the RDF and SPARQL community has adopted methods of SQL innerjoin optimization, and have taken them further with the novel ideas of RDF graph indexing bitmatwww10 (); triad (); rdf3x (); triplebit (). However, optimization of SPARQL queries with other query constructs poses additional challenges, because they restrict the reorderability of the triple patterns across various BGPs. Given below is an OPTIONAL pattern query from atresigmod15 ().
Q1: SELECT ?friend ?sitcom
\lastbox WHERE {
\lastbox :Jerry :hasFriend ?friend .
\lastbox OPTIONAL {
\lastbox ?friend :actedIn ?sitcom .
\lastbox ?sitcom :location :NYC .
\lastbox }
\lastbox }
This query asks for all friends of :Jerry that have acted in a sitcom located in :NYC. In this query, let (:Jerry :hasFriend ?friend) be , (?friend :actedIn ?sitcom) , and (?sitcom :location :NYC) . makes a leftouterjoin over ?friend with . and make an innerjoin between them over ?sitcom. The query can be expressed as . If we consider triple patterns to be equivalent to relational tables, then forms a BGP (say ) with just one triple pattern, and forms another BGP (say ). Note that we emphasized the order of joins by putting the join in a bracket to indicate that this innerjoin must be evaluated before the leftouterjoin between and for the correct results. This is because (we will show this with a toy example in Section 4). Inner and leftouter joins are nonreorderable, so when we have a query with an intermix of other query operators too such as UNIONs, FILTERs in addition to the OPTIONALs, this poses additional restrictions on reorderability. Consider the following query with an intermix of BGPs, OPTIONALs, and UNIONs^{2}^{2}2Unlike SQL, SPARQL standards allows UNIONs between results of different arity..
Q2: SELECT ?friend ?sitcom
\lastbox WHERE {
\lastbox :Jerry :hasFriend ?friend .
\lastbox {
\lastbox {?friend :actedIn ?sitcom .}
\lastbox UNION
\lastbox {?friend :hasFriend ?friend2 .
\lastbox ?friend2 :actedIn ?sitcom .}
\lastbox }
\lastbox OPTIONAL {
\lastbox ?sitcom :hasDirector ?dir .
\lastbox ?sitcom :location :NYC .
\lastbox }}
This query asks for all the friends and friendsoffriends of :Jerry who have acted in any sitcom, and optionally it asks for the directors of the respective sitcoms if their location was NYC. We have in all six triple patterns in this query. Numbering them from top to bottom, the query can be expressed as . These triple patterns form four BGPs in the query, which are as follows: , and then the query can be expressed as . Note that we cannot do the leftouter join between and before evaluating () and the UNION .
Analysis of the real world SPARQL queries shows that queries with an intermix of BGP, OPTIONALs, UNIONs, FILTERs indeed constitute over 94% the query logs usewod11 (); swim (); manvmachine (); practsparql (), and thus make these other constructs like OPTIONAL, UNION, FILTER nonnegligible from the query processing and performance optimization perspective. In our previous work atresigmod15 () we focused on the OPTIONAL pattern queries (referred to henceforth as OPT queries), and proposed novel techniques for the optimization of these queries. Our techniques extended the ideas of nullification and bestmatch (or GeneralizedOuterjoin) operators as proposed in rao2 (); rao1 (); galindolegaria2 (), and showed that for acyclic queries we can reduce the candidate triples to minimal using the semijoin based pruning semij1 (); semij2 (); ullman (), and avoid the nullification and bestmatch operations altogether (see the lemmas in atresigmod15 ()). Since BGPOPT make the building blocks of SPARQL queries, in this paper we mainly show that our BGPOPT optimization techniques can be used as primitives to evaluate queries with an intermix of the various SPARQL query constructs. Much of the research based systems developed for the optimization of SPARQL queries have only handled the BGP component bitmatwww10 (); rdf3x (); triplebit (), and they either do not handle other SPARQL constructs, or rely on naïve ways of processing them. The SPARQL processing systems based on relational databases, such as MonetDB or Virtuoso, just translate the SPARQL queries into their SQL counterparts, by assuming that the RDF graph is stored in the relational tables.
In the light of this, we propose to use our BGPOPT evaluation techniques as primitive building blocks to cover a broader spectrum of the SPARQL queries. While doing so, we focus on the procedural (algorithmic) aspects of using BGPOPT techniques than the empirical aspects, because our previous work has already established the usefulness of our BGPOPT evaluation techniques – especially for the lowselectivity queries^{3}^{3}3Queries which need to process a large amount of data have low selectivity and vice versa.. In this paper, we make the following main contributions.

We propose a new method of forming the Graph of Supernodes (GoSN) that enhances our previously proposed method atresigmod15 () (Section 2).

Using the above mentioned new way of constructing the GoSN, we show that the condition of acyclicity of Graph of Tables (GoT) of a BGPOPT query can be relaxed in some cases in addition to the conditions given in atresigmod15 () (Sections 4.1 and 8.2).

We propose a way of methodically using the BGPOPT query optimization techniques for the queries with an intermix of UNION and FILTER clauses without evaluating each query in the UNION normal form (UNF) individually as proposed in atresigmod15 () (Sections 9.1 and 9.2).

We also discuss handling of the DISTINCT clause with any intermix of these query clauses (Section 9.3).

In the context of UNION and FILTER, we bring to the light implications of “NULLs”, and their semantics for the nullification and bestmatch operators.

Since there is a close match between SPARQL query operators and SQL, our techniques and insights can be useful for their SQL counterparts too, with appropriate indexing and data representation methods in the relational setting.
2 Graph of Supernodes
In our previous work atresigmod15 (), we had outlined a way of capturing a SPARQL query with an intermix of BGP and OPTIONAL patterns using the Graph of Supernodes (GoSN). For the sake of completeness of the text, here we first describe the process of GoSN construction, and then elaborate on the new enhancements. These enhancements help in our propositions regarding the relaxation of the nullification and bestmatch operations, based on the cyclic properties of a query. For this construction of GoSN, we focus only on the BGP OPT patterns without any other SPARQL constructs. They serve as the primitives for applying the BGPOPT query processing techniques for a broader range of queries with any intermix of UNIONs, FILTERs, and DISTINCT as elaborated in Section 9.
A BGPOPT pattern query connects multiple BGP patterns with each other using one or more OPTIONAL keywords. Connecting BGP patterns () with OPTIONAL clauses () introduces restrictions on the order of joining the triple patterns across various BGPs (refer to the example of reorderability given in Section 1). A GoSN is constructed from a BGPOPT query using supernodes, and unidirectional or bidirectional edges, as follows.
Supernodes: In a BGPOPT query of the form , and are patterns that may in turn have nested OPTIONAL patterns inside them, e.g., , or either of and can be OPTfree. Generalizing it, if a pattern does not have any OPTIONAL pattern nested inside it, it is an OPTfree BGP or simply a BGP. From the given nested BGPOPT query, first we extract all such BGPs, and construct a supernode () for each . The triple patterns in are encapsulated in .
Since BGPs are equivalent to SQL innerjoins, and OPTIONAL patterns are equivalent to SQL leftouterjoins, we serialize a nested BGPOPT query considering its BGPs and (innerjoin), (leftouterjoin) operators using proper parentheses. E.g., we serialize Q1 in Section 1 as (), where and are OPTfree BGPs, of encapsulates just , and of encapsulates and (see Figure 2.1).
Unidirectional edges: From the serialized query, we consider each OPT pattern of the type . or may have nested OPTfree BGPs inside them. Using the serializedparenthesized form, we identify the leftmost OPTfree BGPs nested inside and each. E.g., consider the example query given in Fig. 2.2. If , and , and are the leftmost OPTfree BGPs in and respectively, and and are their supernodes. We add a directed edge . If either or does not nest any OPTfree BGPs inside it, we treat the very pattern as the leftmost for adding a directed edge. With this procedure, we can treat OPTIONAL patterns in a query in any order, but for all practical purposes, we start from the innermost OPTIONAL patterns, and recursively go on considering the outer OPTIONAL patterns using the parentheses in the serialized query. E.g., if a serialized query is ( ) (), with as OPTfree BGPs, we add directed edges as follows: (1) , (2) , (3) , (4) .
Bidirectional edges: Next we consider each innerjoin of type in a serialized query. If or has nested OPTIONALs inside, we add a bidirectional edge between the supernodes of leftmost OPTfree BGPs. E.g., if , and , we add a bidirectional edge . If or does not nest any OPTIONALs inside it, we consider the very pattern to be the leftmost for adding a bidirectional edge. We add bidirectional edges starting from the innermost innerjoins () using the parentheses in the serialized query, and recursively go on considering the outer ones, until no more bidirectional edges can be added. Considering the same example given under unidirectional edges, we add a bidirectional edge between . The graph of supernodes (GoSN) for this example is shown in Figure 2.2. Thus we completely capture the nesting of BGPs and OPTIONALs in a query using this GoSN.
2.1 Nomenclature
Next, we introduce nomenclatures with respect to the supernodes in a GoSN and the OPTIONAL patterns in a SPARQL query.
MasterSlave: In an OPTIONAL pattern , we call pattern to be a master of , and a slave of . This masterslave relationship is transitive, i.e., if a supernode is reachable from another supernode by following at least one unidirectional edge in GoSN (), then is called a master of (see Figure 2.2).
Peers: We call two supernodes to be peers if they are connected to each other through a bidirectional edge, or they can be reached from each other by following only bidirectional edges in GoSN, e.g., and in Figure 2.2.
Absolute masters: Supernodes that are not reachable from any other supernode through a path involving any unidirectional edge are called the absolute masters, e.g., and in Figure 2.2 are absolute masters.
These masterslave, peer, and absolute master nomenclatures and relationships apply to any triple patterns enclosed within the respective supernodes too.
Welldesigned patterns: As per the definition given by Pérez et al, a welldesigned OPT query is – for every subpattern of type in the query, if a join variable in appears outside , then also appears in . A query that violates this condition is said to be nonwelldesigned.
For the scope of the text in this paper, we mainly consider welldesigned queries, because they occur most commonly for RDF graphs, and remain unaffected by the differences between SPARQL and SQL semantics over the treatment of NULLs. Our previous text atresigmod15 () discusses nonwelldesigned queries and their effect on the treatment of NULLs. We request the interested readers to refer to those (please see Appendices B and C in atresigmod15 ()).
2.2 Graph of Triple Patterns (GoT)
During the GoSN construction, we only added connections between the supernodes formed out of the BGPs in a query based on the structural semantics of the given query. Next we add labeled undirected edges between triple patterns as follows. If two triple patterns share one or more join variables among them, and are in direct masterslave hierarchy, or are part of the same supernode, we add an undirected edge between them. For instance, let and share a join variable . If , or if , then we add an undirected edge between and , with the edge label . Recall that the triple patterns encapsulated in the supernodes share the same masterslave or peer hierarchy as their respective supernodes. These undirected edges among the triple patterns create a graph of triple patterns (GoT) atresigmod15 (). The GoT for Q1 in Section 1 is shown by “red” connecting edges in Figure 2.1. The edge labels in the GoT are not shown to avoid cluttering the figure.
Definition 2.1
If the graph of tables (GoT) of a BGPOPT query is connected, then the query is free from any Cartesian joins, and is considered to be a connected query.
E.g., following is an example of a nonconnected query (Cartesian join), because the triple pattern (?actor :livesIn :LA) does not have any shared variable with the other two triple patterns (:Jerry :hasFriend ?friend) and (?friend :name ?name).
SELECT ?friend ?name ?actor
WHERE {
\lastbox :Jerry :hasFriend ?friend .
\lastbox ?friend :name ?name .
\lastbox OPTIONAL {
\lastbox ?actor :livesIn :LA .
\lastbox }}
Let us consider a subgraph of this GoT consisting of only the triple patterns encapsulated inside all the absolute master supernodes and all the undirected edges incident on them.
Property 2.1
If a query is welldesigned and connected, then the subgraph of GoT consisting only of triple patterns within the absolute masters is always connected.
Property 2.2
In a welldesigned connected query, a slave supernode never has more than one incoming unidirectional edge.
Observing Properties 2.1 and 2.2, we coalesce all the absolute masters of GoSN to form a single absolute master supernode of the GoSN – . While doing so, we remove any bidirectional edges incident on the coalesced absolute master supernodes. For every unidirectional edge between a coalesced absolute master and its slave, a unidirectional edge is added between and the corresponding slave. Figure 2.3 shows the transformed GoSN of the original GoSN shown in Figure 2.2, after coalescing absolute masters and .
3 Acyclicity and Minimality
In this section, first we define the acyclicity of a SPARQL (equivalently SQL) query, and then discuss the minimality of triples (tuples), and the effect of the cyclic properties of a query on it.
3.1 Acyclicity of Queries
For the concept of acyclicity of a query we take into consideration the graph of tables (GoT). We define the equivalence classes of edges of GoT as follows.
Definition 3.1
For every triple pattern, its incident edges in GoT are put in equivalence classes such that all the edges in a given equivalence class have either same edge labels or their edge labels are subset of some other edge’s label in the same class. E.g., if has three edges incident on it with labels , , , then make one equivalence class and in another.
A triple pattern is called a leaf if it has only one equivalence class among its incident edges. An acyclic query is then defined as follows. If we recursively remove leaf triple patterns, and edges incident on them from a GoT, and then if we are left with an empty GoT at the end, then the query is acyclic. The set of leaf triple patterns are chosen recursively in each round after previous leaves and their incident edges are removed. This process is reminiscent of GYOreduction ullman (). GYOreduction assumes a hypergraph where each attribute in a table is a node, and a hyperedge represents a table. However, to be consistent with our representation of GoT and GoSN, we have formulated this definition of acyclicity instead of using GYOreduction.
3.2 Minimality of triples
The triples associated with a triple pattern (or tuples in a table) are said to be minimal for the given join (BGP or BGPOPT) query, if every triple (tuple) is part of one or more final join results of the query. There does not exist any triple that gets eliminated as a result of its join with another triple (associated with another triple pattern). Consider the same query given in Figure 2.1, along with the sample data associated with it in Figure 4.1. ?friend :actedIn ?sitcom has five triples associated with it – (1) :Larry :actedIn :CurbYourEnthu, (2) :Julia :actedIn :Seinfeld, (3) :Julia :actedIn :Veep, (4) :Julia :actedIn :NewAdvOldChristine, (5) :Julia :actedIn :CurbYourEnthu. But they are not minimal for this query, because after ’s join with (?sitcom :location :NYC), all tuples but :Julia :actedIn :Seinfeld associated with get eliminated.
4 Nullification and Bestmatch
SPARQL BGP queries are analogous to the SQL innerjoin queries, and hence the joins over the triple patterns in BGP queries are associative and commutative, i.e., a change in the order of joins between the triple patterns does not change the query results. SPARQL queries with OPTIONAL patterns are analogous to the SQL leftouterjoins, and hence they are not associative or commutative. An example of such nonreorderable query is given in Section 1. However, reorderability of joins is a powerful feature that enables the query optimizer to explore many query plans. Hence, Rao et al and GalindoLegaria, Rosenthal proposed ways of reordering intermixed inner and leftouter joins by using additional operators nullification and bestmatch (or Generalized Outerjoin) galindolegaria1 (); galindolegaria2 (); rao2 (); rao1 ().
For the completeness of the text, first we will briefly see how nullification and bestmatch operators work with the same example as given in atresigmod15 (). For more details of these operators, we request the interested readers to refer to rao1 (); rao2 (); galindolegaria1 (); galindolegaria2 (). Consider the same query given in Figure 2.1, along with the sample data associated with it in Figure 4.1. :NYC has been the location for a lot of American sitcoms, and a lot of actors have acted in them (they are not shown in the sample data for conciseness). But, among all such actors, :Jerry has only two friends, :Julia and :Larry. Hence, is more selective than and . A leftouterjoin reordering algorithm as proposed in rao1 (); galindolegaria2 () will typically reorder these joins as . Due to this reordering, all four sitcoms that :Julia has acted in show up as the bindings of ?sitcom (see Res1 in Fig. 4.1), although only :Seinfeld was located in the :NYC. To fix this, nullification operator is used, which ensures that variable bindings across the reordered joins are consistent with the original join order in the query (see Res2 in Figure 4.1).
The nullification operation caused results that are subsumed within other results. A result is said to be subsumed within another result (), if for every nonnull variable binding in , has the same binding, and has more nonnull variable bindings than . Thus results 3–5 in Res2 are subsumed within result 2. The bestmatch operator (or mimumum union as defined by GalindoLegaria in galindosigmod ()) removes all the subsumed results (see Res3). Final results of the query are given as bestmatch(nullification()).
4.1 Nullification, Bestmatch, and Minimality of triples
In atresigmod15 (), we made an important observation that if every triple pattern in an OPTIONAL pattern query has minimal triples associated with it, then nullification and bestmatch operations are not required (ref. Lemma 3.1 in atresigmod15 ()). We made yet another observation through Lemmas 3.3 and 3.4 in atresigmod15 (), that for the following two classes of the OPTIONAL pattern queries nullification and bestmatch is not required if we use Algorithm1 in atresigmod15 () to reduce the set of triples associated with each triple pattern in the query, prior to generating final results using multiway pipeline join algorithm (Algorithm3 in atresigmod15 ()).

Acyclic GoT: OPTIONAL pattern queries whose GoT is acyclic.

Only one join variable per slave: OPTIONAL pattern queries where there are cycles in the GoT, but any slave triple pattern has only one join variable in it, and that join variable is shared with its master triple pattern.
These classes of OPTIONAL pattern queries are considered good because they can avoid the overheads of the nullification and bestmatch operations despite the reordering of inner and leftouter joins. The important premise of these observations is that Algorithm1 reduces the triples associated with each triple pattern in the OPTIONAL pattern query in such a way that even if we reorder the inner and leftouter joins while doing the multiway pipelined join, it does not generate spurious results, and thus avoids the necessity of nullification and bestmatch. We extend this class of good OPTIONAL pattern cyclic queries beyond the ones in which all slaves have only one join variable, and these are discussed in Section 8.2. Before that we revisit our pruning method and multiway joins to work with just GoT, and obviate the need of graph of join variables (GoJ) that was used in our previous work atresigmod15 ().
5 Pruning Triples
SPARQL (in turn SQL) queries can be evaluated using different equivalent plans. All the plans output exact same results. Typically, a plan with the least cost is chosen for evaluation. In the previous sections, we established the relationship between structural properties of a BGPOPT query, minimality of triples, and nullification, bestmatch operations. However, we get the benefit of avoiding nullification and bestmatch, only if the tuples associated with the query before performing the reordered joins are in the minimal (or favorable) form. We ensure that by using the pruning step before performing joins. The pruning phase only prunes the triples associated with each triple pattern in the query using a series of semijoins, and the multiway pipelined joins then produce the join results in a pipelined fashion, followed by nullification and bestmatch if required.
5.1 Semijoins
Pruning of triples without performing joins is achieved through semijoins. Semijoins can be notationally represented as follows. . Here is a triple matching , and is a variable binding (value) of variable in . After this semijoin, is left with only triples whose bindings are also in , and all other triples are removed.
Bernstein et al semij1 (); semij2 () and Ullman ullman () have proved previously that if the graph of tables (GoT) of an innerjoin query is acyclic, a bottomup followed by a topdown pass with semijoins at each table in this tree, reduces the set of tuples in each table to a minimal. Note that for the discussion of minimality of triples and semijoins, we focus on the GoT of a query, and not GoSN. GoSN inherently encapsulates GoT inside it, since GoSN maintains connections between BGP blocks, whereas GoT maintains connections between individual triple patterns.
In our previous work, we proposed a pruning algorithm for BGPOPT queries that makes use of graph of join variables (GoJ) and clusteredsemijoins. However, in this paper we propose an improved algorithm. Our algorithm is reminiscent of the full reducer semijoin sequence as given in ullman (), that uses the concept of graphs with hyperedges to represent tables (triple patterns for SPARQL). But fullreducers only addressed the innerjoins, and we address an intermix of inner and leftouter joins. We first discuss it in Algorithm1, and then discuss its differences from our previous Algorithm1 in atresigmod15 ().
1. The working of the algorithm is described as follows. We first order all the supernodes in the GoSN of a query according to the masterslave hierarchy – masters before their respective slaves, and among any two supernodes not in the masterslave hierarchy, we randomly pick one over another. Note that since these are welldesigned queries and we have coalesced all the absolute master supernodes into , the relative ordering among two such supernodes not in the masterslave hierarchy does not matter. We call this ordering among the supernodes snorder as given by the ordersupernodes function (ln 1), and appears first in this order. If GoT is cyclic and there are cycles in slave supernodes too, then we just do pruning by following a greedy order of semijoins – doing semijoins over highly selective triple patterns first – honoring the masterslave hierarchy among the triple patterns (1–1), and end the pruning process there. In this case nullification and bestmatch are necessary after multiwayjoins. However, if GoT is cyclic but the cycles are confined only to , and GoTs of slaves are acyclic, we consider a greedy order of semijoins over the triple patterns in , prune the triples in , and remove from snorder (ln 1–1).
Then, starting with the next supernode, , in snorder (ln 1) we consider all the triple patterns encapsulated within that supernode. For each triple pattern in , and its connected triple patterns in alone, we create equivalence classes of edges of GoT incident on (recall the definition of equivalence class of edges given in Section 3). We do not consider edges that connect with triple patterns outside . After doing this for all the triple patterns in , we pick the triple patterns that have only one equivalence class among its edges (ln 1). This triple pattern is a leaf node. Among all such leaf nodes, we pick the one which has the least number of triples associated with it – most selective one. In case of a tie, we pick one randomly (ln 1).
Among leaf ’s connected triple patterns within , we pick the neighbor with the least number of triples associated with it, say (ln 1), and add a semijoin to the queue (bottomup semijoin order) (ln 1). If is a slave supernode, we ensure to fetch its master’s variable binding restrictions as follows. If is a master of , such that , we transfer the constraints on the variable bindings from to as follows. Without losing generality, while adding a semijoin between two triple patterns in to , we first check if has a neighbor in . If it does, then we add to before . This ensures that any variable bindings imposed by a master of the triple patterns are transferred to their respective slave triple patterns (ln 1). Next we remove and all the edges incident on it from consideration, and repeat the same procedure described above with the rest of the triple patterns in .
Notice that through this procedure we recursively define a spanning tree over the graph of triple patterns (GoT) encapsulated within , and make a bottomup pass over it – every semijoin denotes to be the parent of in the spanning tree, and the semijoin denotes the direction of the walk from the leaves to the internal nodes and the root of the tree, e.g., semijoin denotes a walk from on the spanning tree. For the topdown pass, we simply reverse the queue to get (ln 1). That is for every semijoin of type in , we add to . However, we omit all the semijoins of type where is a master of (ln 1). When the GoT is acyclic, the very first is always .
The main differences between our previously proposed technique (ref Algorithms 3.1, 3.2 in atresigmod15 ()), and Algorithm 1 here are as follows:

In the present technique, we form a rooted tree over GoT, whereas in atresigmod15 (), the rooted tree was formed over graph of join variables (GoJ).

In our present technique the rooted tree over GoT is formed in a bottomup fashion, where we first pick the most selective triple patterns as the leaves and recursively build the internal nodes and root of the tree. In atresigmod15 (), the rooted tree over GoJ was formed in a topdown fashion, where we first picked a join variable with lowest selectivity as the root of the tree, and then recursively picked the internal nodes and leaves.

In atresigmod15 (), we first did a bottomup pass over all the induced GoJs of individual supernodes, and then did the topdown passes. In the present technique, we do the bottomup and topdown passes on the triple patterns of each supernode in one go. Thus we do the full possible pruning of the triples associated with the triple patterns in a supernode, before moving on to its slaves. This helps in better pruning of the slaves, because their respective masters are already in the pruned state.
Our discussion of the properties of nullification, cyclicity of queries, minimality of triples, and the pruning procedure until now has been agnostic to the lower level storage structure and indexes on the RDF graph. Indeed, our propositions and algorithms presented earlier can work on any storage and index structure – only the semijoins procedure used in Algorithm1 will change depending on the storage structure and indexes. Nevertheless, an efficient storage and index structure helps in achieving better performance. We achieve this through the usage of compressed bitvector indexes on RDF graphs, and procedures that directly work on these compressed indexes without decompressing them as proposed in bitmatwww10 (); atresigmod15 (). For the completeness of the text, these are described in Section 6.
6 BitMat Indexes
In an RDF graph, let , , and be the sets of unique subject, predicate, and object values. Then a 3D bitcube of RDF data has dimensions. Each cell in this bitcube represents a unique RDF triple formed by the coordinate values (S P O). If this (S P O) triple is present in the given RDF dataset, that bit is set to 1 in the bitcube. The unique values of subjects, predicates, and objects in the original RDF data are first mapped to integer IDs, which in turn are mapped to the bitcube dimensions. To facilitate joins on SO dimensions, same S and O values are mapped to the same coordinates of the respective dimensions^{4}^{4}4For the scope of this paper, we do not consider joins on SP or OP dimensions..
Let . Set is mapped to a sequence of integers 1 to . Set is mapped to a sequence of integers to . Set is mapped to a sequence of integers to , and set is mapped to a sequence of integers 1 to .
This bitcube is conceptually sliced along each dimension, and the 2D BitMats are created. In general, four types of 2D BitMats are created: (1) SO and OS BitMats by slicing the Pdimension (OS BitMats are nothing but a transpose of the respective SO BitMats), (2) PO BitMats by slicing the Sdimension, and (3) PS BitMats by slicing the Odimension. Altogether we store BitMats for any RDF data. Figure 6.1 shows 2D SO BitMats that we can get by slicing the predicate dimension (others are not shown for conciseness).
Each row of these 2D BitMats is compressed as follows. In the runlengthencoding, a bitrow like “1110011110” is represented as “[1] 3 2 4 1”, and “0010010000” is represented as “[0] 2 1 2 1 4”. Notably, in the second case, the bitrow has only two set bits, but it has to use five integers in the compressed representation. So we use a hybrid representation in our implementation that works as follows – if the number of set bits in a bitrow are less than the number of integers used to represent it, then we simply store the set bit positions. So “0010010000” will be compressed as “3 6” (3 and 6 being the positions of the set bits). This hybrid compression fetches us as much as 40% reduction in the index space compared to using only runlengthencoding as done in bitmatsrc ().
Other metainformation such as, the number of triples, and condensed representation of all the nonempty rows and columns in each BitMat, is also stored along with each BitMat. This information helps us in quickly determining the number of triples in each BitMat and its selectivity without counting each triple in it, while processing the queries. A 2D SO or OS BitMat of predicate :hasFriend represents all the triples matching a triple pattern of kind (?a :hasFriend ?b), a 2D PS BitMat of Ovalue :Seinfeld represents all triples matching triple pattern (?c ?d :Seinfeld), and so on.
Query execution uses the fold and unfold primitives, which process the compressed BitMats without uncompressing them bitmatwww10 ().
Fold operation is represented as ‘fold(BitMat,
Re
tainDimension) returns bitArray’. It takes a 2D BitMat and
folds it by retaining the RetainDimension.
More succinctly, a fold operation is nothing but projection of
the distinct values of the particular BitMat dimension, by doing a bitwise OR
on the other dimension. It can be represented as:
is a 2D BitMat holding the triples matching , and is the dimension of BitMat that represents variable in . E.g., for a triple pattern (?friend :actedIn ?sitcom), if we consider the OS BitMat of predicate :actedIn, ?friend values are in the column dimension, and ?sitcom values are in the row dimension of the BitMat.
Unfold is represented as ‘unfold(BitMat,
MaskBit
Array, RetainDimension)’. For every bit set to 0 in the
MaskBitArray, unfold clears all the bits corresponding to that
position of the RetainDimension of the BitMat. Unfold can be simply
represented as:
is a triple in that matches . is the MaskBitArray containing bindings of to be retained. is the dimension of that represents , and is a binding of in triple . In short, unfold keeps only those triples whose respective bindings of are set to 1 in , and removes all other.
Until now we discussed semijoins only notationally as . Semijoins can be implemented using the BitMat indexes and the fold, unfold primitives as given in Algorithm 2.
7 Multiway Pipelined Joins
In the previous sections, we established the algorithmic basis of our techniques for understanding the characteristics of a BGPOPT query and pruning the triples associated with the each triple pattern in the query before generating the final results – Algorithm 1 only prunes the candidate RDF triples, but does not generate the final results. In Section 6, we took an overview of our storage and indexing structure for the RDF graphs. In this section, put the pruning algorithm and index structure together with our technique of multiwaypipelined joins to generate the final results as given in Algorithm3.
In Algorithm 3 for query processing, we first construct the GoSN (ln 3). Next, we decide if nullification and bestmatch are required. This decision is based on the cyclic properties the query, and we discuss them in Section 8, and Lemmas 8.1 and 8.2. We call prune_triples (Algorithm1) to prune the unwanted triples. Pruning operation also onthefly loads the BitMat associated with each triple pattern containing only triples satisfying that triple pattern (we have not shown this operation explicitly in Algorithm1, but is explained next). We choose an appropriate BitMat for each triple pattern as follows. If the triple pattern in the query is of type (?var :fx1 :fx2), i.e., with two fixed positions, we load only one row corresponding to :fx1 from the PS BitMat for :fx2. Similarly for a TP of type (:fx1 :fx2 ?var), we load only one row corresponding to :fx2 from the PO BitMat for :fx1. E.g., for (?sitcom :location :NYC) we load only one row corresponding to :location from the PS BitMat of :NYC. If the triple pattern is of type (?var1 :fx1 ?var2), we load either the SO or OS BitMat of :fx1. If ?var1 is a join variable and ?var2 is not, we load the SO BitMat and vice versa. If both, ?var1 and ?var2, are join variables, we observe the order of semijoins in to check if the semijoin over the given triple pattern is over ?var1 or ?var2 first. If semijoin over ?var1 comes before ?var2, we load the SO BitMat and vice versa.
While loading the BitMats, we do active pruning using the triple patterns whose BitMats are already initialized. E.g., if we first load BitMat containing triples matching (:Jerry :hasFriend ?friend), then while loading , we use the bindings of ?friend in to actively prune the triples in while loading it. Then while loading , we use the bindings of ?sitcom in to actively prune the triples in . We check whether two triple patterns are joining with each other over an inner or leftouter join using GoSN with the masterslave or peer relationship, and then we decide whether to use other BitMat’s variable bindings.
Note that using prune_triples, we prune the triples in BitMats, but we need to actually “join” them to produce the final results. For that we use multiwayjoin (ln 3 in Algorithm 3). This procedure is described separately in Section 7.1. After multiwayjoin, we use bestmatch to remove any subsumed results if nullification was required as a part of multiwayjoin (discussed in Section 8). In bestmatch, we externally sort all the results generated by multiwayjoin, and then remove the subsumed results with a single pass over them.
7.1 Multiway Pipelined Join
Before calling multiwayjoin, we first sort all the triple patterns in the query as follows. Considering the triple patterns in , we sort them in the ascending order of the number of triples left in each triple pattern’s BitMat. Then we sort the remaining supernodes in the descending order of masterslave hierarchy. That is, among two supernodes connected as , triple patterns in are sorted before those in . Among the triple patterns in the same supernode (peers), they are sorted in the ascending order of the number of triples left in their BitMats. Let us call this order tporder (ln 3 in Algorithm 3). From tporder, we construct a conceptual spanning tree as follows. The very first triple pattern is designated as the root of the tree, and added to a new sort order of triple patterns, called stps. We recursively pick the next triple pattern from tporder such that it is connected to at least one triple pattern in stps (ln 3 in Algorithm 3). This stps is used in multiwayjoin to produce final results and decide the join order. In multiwayjoin we also use at most additional memory buffer, where are the variables in every triple pattern in the query . This is vmap in Alg 4. Thus we use negligible additional memory in multiwayjoin.
At the beginning, multiwayjoin gets stps, an empty vmap for storing the variable bindings, an empty visited list, and a flag nulreqd indicating if nullification is required. We go over each triple in of the first triple pattern in stps, generate bindings for the variables in , and store them in vmap. We add to the visited list, and call multiwayjoin recursively for the rest of the triple patterns (ln 4–4). In each recursive call, multiwayjoin gets a partially populated vmap and a visited list that tells which triple pattern’s variable bindings are already stored in vmap. Stps order is formed in a way that from second position onward each triple pattern in stps has at least one connection in stps order before it. Thus, for the next recursive call of multiwayjoin, we are expected to find at least one variable binding in vmap for the next nonvisited (ln LABEL:ln:getbind). Stps order ensures that a master triple pattern’s variable bindings are stored in vmap before its slaves. If there exists one or more triples in consistent with the variable bindings in vmap, then for each such we generate bindings for all the variables in , store them in vmap, and proceed with the recursive call to multiwayjoin for the rest of the triple patterns (ln 4–4). Notice that, this way we pipeline all the BitMats, and do not use any other intermediate storage like hashtables.
If we do not find any triple in consistent with the existing variable bindings in vmap, then – (1) if is an absolute master, we rollback from this point, because an absolute master triple pattern cannot have NULL bindings (ln 4), else (2) we map all the variables in to NULLs, and proceed with the recursive call to multiwayjoin (ln 4–4). When all the triple patterns in the query are in the visited list, we check if we require nullification to ensure consistent variable bindings in vmap across all the slave triple patterns, and output one result (ln 4–4). We continue this recursive procedure till triples in are exhausted (ln 4–4). Intuitively, multiwayjoin is reminiscent of a relational join plan with reordered leftouterjoins – that is, in stps we sort master triple patterns before their slaves, and masters generate variable bindings before slaves in vmap.
8 Cycles in the Queries
In this section we discuss the cyclic properties of the queries and how they affect the requirement of nullification and bestmatch operations after reordering the inner and leftouterjoins in a query, that makes an important part of reorderability of joins.
8.1 Acyclic Queries
In atresigmod15 (), we proposed our pruning algorithm for OPTIONAL pattern queries by constructing a graph of join variables (GoJ) as follows. Every join variable in the query is a node in GoJ, and two join variables have an undirected edge between them, if both the join variables cooccur in a triple pattern in the query. In atresigmod15 (), we proposed that if the GoT of an OPT query is acyclic then the GoJ is acyclic too. However, there exists queries where this equivalence may not hold. An example of such a query is given below.
SELECT ?a ?b ?c
WHERE {
\lastbox ?a :p1 ?b .
\lastbox ?b :p2 ?c .
\lastbox ?c :p3 ?a .
\lastbox ?a ?b ?c .
}
Note that in this query the triple pattern (?a ?b ?c) joins with every other triple pattern over two join variables. Per our definition of acyclicity given in Section 3.1, this query is indeed acyclic. However, its GoJ is cyclic as shown in Figure 8.1. Also note that in this query there is a join on the objectpredicate position ( and ) which is very uncommon in the RDF data. Most SPARQL queries have triple patterns joining over only one variable, and these joins are on subjectobject, subjectsubject, or objectobject positions (since they naturally indicate edges incident on the nodes in the graph). Hence our original proposition and the technique of clusteredsemijoins on GoJ in atresigmod15 () still work correctly for the SPARQL queries where any two triple patterns always join only over one join variable (and all the queries used in our experiments in atresigmod15 () satisfied this condition). Nevertheless, our improved prune_triples method (Algorithm 1) in this article works correctly for such corner case queries too.
Lemma 8.1
Nullification and bestmatch can be avoided for an OPTIONAL pattern query with an acyclic GoT.
Proof
Nullification and bestmatch processes are described in Section 4. In that, it can be observed that nullification is required when some variable in a triple pattern is bound to a value that does not exist in another triple pattern that is either master or peer. When a BGPOPT query is completely acyclic – individual GoTs of each OPTfree BGP component as well as the entire GoT of the query across all the triple patterns is acyclic – prune_triples (Algorithm 1) ensures that each triple pattern is left with minimal number of triples – there does not exist any binding for a variable in that does not exist in its master or peer triple pattern .
This minimality is ensured as follows. When we do pruning of triples in
, the triples associated with triple patterns in are
reduced to minimal. This follows directly from the proofs of Bernstein et al.
and Ullman semij1 (); semij2 (); ullman (). When we do pruning of triples in slave
supernodes, the bindings in its acyclic master supernode are already
minimalized, and we propagate those on the present slave supernode’s
variable bindings (lines 1–1 in
Algorithm1). Thus after completion of prune
_triples,
each triple pattern in the query has minimal triples associated with its triple
patterns. Thus nullification and bestmatch can be avoided. ∎
8.2 Cyclic Queries
For OPTfree BGPs, i.e., pure innerjoins, with a cyclic GoT, minimality of triples cannot be guaranteed using the procedure of bottomup followed by topdown pass of semijoins on the spanning tree over GoT as described in in Section 8.1 semij1 (); semij2 (); ullman (). This result carries over immediately to the queries with an intermix of BGP and OPT patterns too whose GoT is cyclic. Thus, we simply prune the triples by following a greedy order of semijoins, while adhering to the masterslave hierarchy among the triple patterns in the query. The greedy order of semijoins is determined by the relative selectivity of the triple patterns, and the masterslave hierarchy between them (ln 1 in Algorithm 1). Since minimality of triples in each TP is not guaranteed, we need to use the nullification and bestmatch operations in a reordered query to ensure consistent variable bindings, and to remove subsumed results.
This observation in general holds for all cyclic BGPOPT queries, but we identify a subclass of cyclic BGPOPT queries that can avoid nullification and bestmatch – in such queries the following conditions hold:

A subgraph of GoT representing only the triple patterns in any slave supernode is acyclic.

There is only equivalence class of edges connecting triple patterns in the masterslave hierarchy between two supernodes. That is, if , and we put all GoT edges of type in equivalence classes, then there is only one equivalence class. Generalizing, this property holds for each and every pair of supernodes in masterslave hierarchy that have an directed edge among them in GoSN. Recall our definition of equivalence classes of edges given in Definition 3.1.
For such queries, we do greedy way of pruning using semijoins over the triple patterns in , and after that for each slave supernode, we follow the same procedure of bottomup and topdown pruning as described in Algorithm 1 (ref. ln 1–1 and ln 1–1).
In Figure 8.2, we have given an example of a query where there are two equivalence classes of GoT edges running between and the slave supernode , and hence it cannot avoid nullification and bestmatch. Notice carefully that in this query although the individual GoTs of and the slave supernode are acyclic, the GoT over all the triple patterns is cyclic.
Lemma 8.2
For a BGPOPT query, if (1) there is only one equivalence class of edges across any pair of masterslave supernodes, and (2) each subgraph of GoT representing the triple patterns in each slave supernode is acyclic, then nullification and bestmatch can be avoided if the triples associated with the triple patterns are pruned with prune_triples (Algorithm 1), and results are produced using multiwayjoin (Algorithm 4). This holds even if the GoT of the triple patterns in is cyclic.
Proof
In Lemma 8.1, we saw that if each OPTfree BGP component of a
query has triple patterns with minimal triples associated with it,
nullification and bestmatch can be avoided. When has a cyclic GoT,
it cannot be guaranteed that the triple patterns in it have minimal triples
associated with them after prune_triples method. Note that we prune
the triples in before any other slave supernodes in the query, and
then do not revisit it again in prune_triples (lines
1 and 1). Thus when a slave supernode fetches
the variable bindings from its master (lines
11 in Algorithm 1), the master’s
variable bindings have already been frozen (irrespective of whether minimal or
not). During multi
wayjoin, we again visit triple patterns in their
masterslave hierarchy, i.e., we start with the triple patterns in ,
create variable bindings for all the triple patterns in it, and then move on to
other slave supernodes, in their respective masterslave hierarchy.
If the GoT of just triple patterns in has cycles, then we backtrack while generating variable bindings for its triple patterns whenever there is a mismatch (ln 4 in Algorithm 4). Thus when we start processing the slave triple patterns from an acyclic slave supernode, all the variable bindings of have been decided, and are consistent across all the triple patterns of . Having only one equivalence class of edges between and its slave, say , means that there is always one triple pattern, say , in and respectively in , which have one or more shared variables between them that cover all the shared variables between and . Then in multiwayjoin, whenever we start processing the triple patterns from for generating variable bindings, we first ensure that in has the exact same bindings for all the variables shared between it and . It may happen that such a (composite) binding does not exist in , then we immediately set all the variables in to NULL bindings, and do the same for all of slaves of too. Thus at the end of one iteration of multiwayjoin, we do not have any inconsistent variable bindings in vmap that necessitate nullification, followed by bestmatch. We assume that the BGPOPT queries are welldesigned, which ensures that no masterslave supernode that is not connected with a directed edge share variables among them, that are not shared between any intermediate master supernode.
Note that this is possible only because each slave supernode has an acyclic GoT, and the triple patterns in it have minimal triples, that are consistent across the same slave supernode. Since there is only one equivalence class edge between any masterslave pair of supernodes, a unique triple pattern in the slave can determine if the respective bindings exist in the slave. ∎
9 UNIONs, FILTERs, DISTINCT
Basic Graph Patterns (BGPs) connect the triple patterns through innerjoins, and the OPTIONAL patterns connect two BGPs through leftouterjoins. BGPOPT patterns make the basic building blocks of the SPARQL query language, as we will see in this section later through the query rewriting rules in UNIONs and FILTERs. SPARQL, like SQL, has other nonjoin constructs too such as UNIONs, FILTERs, DISTINCT. SPARQL also has additional constructs like ORDER BY, FROM, FROM NAMED, REDUCED, OFFSET, LIMIT, ASK, CONSTRUCT etc. SPARQL grammar is recursive, and it can be pictorially represented as shown in Figure 9.1^{5}^{5}5For the clarity and conciseness, in Figure 9.1 we have shown the core recursive performance intensive components of SPARQL. SPARQL 1.1 grammar has other syntactic components that are not shown in the figure.. In our work we have mainly focused on the SPARQL components that constitute the recursive part of the language, and thus make the performance intensive components of the query evaluation strategies. Having done the BGPOPT component analysis in the previous sections, further in this section, we analyze UNIONs, FILTERs, and DISTINCT clauses, and their interaction with the BGPOPT component.
9.1 UNIONs
The SPARQL grammar does not enforce the two patterns being UNIONed to be union compatible (SQL does)^{6}^{6}6Two patterns are union compatible, if they have the same arity and same attributes, e.g., for SPARQL it means that the two patterns would have the same size and the same variables in them.. However, for our discussion, we assume welldesigned UNIONs – for every subpattern in a query, if a variable appears outside , then it must appear both in and . This assumption avoids having to deal with a query with “dangerous” variables, Cartesian products, and disconnected GoT polleres ().
For any BGPOPTUNION query, first we identify UNIONfree BGPOPT subcomponents of the query. E.g., in a query of the form , where are all OPTfree BGPs, and following are the UNIONfree components – (a) , (b) , (c) . For each such UNIONfree BGPOPT subcomponent of the query, we prune the triples associated with each triple pattern in that component using our prune_triples procedure (Algorithm 1). Note that for this pruning procedure, we ignore all the other subcomponents of the query and the respective triple patterns in those subcomponents. We only consider the given subcomponent as an independent BGPOPT query.
Next we apply following three conversion rules on the BGPOPTUNION query and convert the query in the UNION Normal Form (UNF) perez2 (). Note that for the notations given below a pattern need not be a BGP, it can be a pattern with other subpatterns nested inside it, and the symbol indicates that the results generated by the queries on the either side of are always the same for a welldesigned union compatible query.

.


is rewritten as . However, for conventional union^{7}^{7}7However, if we do a minimumunion the results will be the same as elaborated later.. We will elaborate on this rewrite after presenting our query evaluation technique below.
We convert any BGPOPTUNION query in the UNF by applying the above three conversion rules, such that the resulting query looks like where each is a UNIONfree BGPOPT pattern. Note that we have deliberately used suffixes for this representation, to avoid confusion with the patterns in a BGPOPTUNION query before bringing it in the UNF. Now each of these subqueries can be treated independently as BGPOPT queries for the purpose of entire query evaluation. For each such subquery, we run multiwayjoin as given in Algorithm 4. Note that multiwayjoin needs to know if it has to do nullification on each generated result. Just like we do for any UNIONfree BGPOPT query, for each subquery in the UNF, nullification is required if it violates any of the following conditions:

Considering the GoSN of , GoT of triple patterns enclosed in each slave of GoSN is acyclic.

For every pair of masterslave supernodes such as , we consider equivalence classes of GoT edges (recall our definition of equivalent classes of GoT edges from Section 5). Then for every pair of masterslave supernodes, there is only one equivalence class of GoT edges between masterslave triple patterns.
Once the final results are generated for each subquery in the UNF using the multiwayjoin and nullification (wherever required), we do a “union all” of results of all the subqueries – all the results are compiled together along with any duplicates as well. Next, we decide when we need to use the bestmatch operation over the unioned results, and after that we elaborate why we do this. Bestmatch is required if –

nullification operation was used in at least one subquery in the UNF.

while bringing the original query in UNF, at least once pattern was rewritten as .
For the first condition – nullification used in any subqueries – it is straightforward to see that bestmatch is required to remove the subsumed results (ref. Section 4). For the second condition, recall that for the third union expansion rule, we noted that , and now we elaborate on this. Per SPARQL grammar, the UNION operation does a “union all” of the results produced by the unioned patterns, without removing the duplicates. Also when the unioned elements have arity, the set union operation does not remove subsumed results (ref. Section 4 for the definition of subsumed results.).
Definition 9.1
A union operation that removes subsumed results is called minimum union.
Let two sets of bindings for variable pairs be unioned as {(:p1, NULL)} {(:p1, :p2)}. The result of this union is {(:p1, NULL), (:p1, :p2)}. However, if the sets of variable bindings being unioned are {(NULL, NULL)} {(:p1, :p2)}. Then the result is {(:p1, :p2)}. This means that union of NULL coexisting with another nonnull value stays in the final results despite being subsumed by another result, but not by itself. Also in SPARQL, unlike most SQL systems, the joins are nullcompatible, i.e., the join of variable bindings of , is , and . Because of the above two important observations, if we rewrite a pattern as , it may generate different results than , if either or or both generate all null results.
Thus, for pattern rewritten as , the final results may not match. However the original query and its UNF rewrite are semantically the same, and a query execution method that employs minimumunion instead of unionall or setunion generates all the nonduplicate and nonsubsumed results correctly.
Thus we conclude this discussion that minimum union instead of setunion or unionall, allows the third rewrite rule for converting a BGPOPTUNION query into UNF while keeping the patterns before and after the rewrite equivalent with respect to the generated results. As a result, it allows any BGPOPTUNION query to be brought in the UNF allowing us more options for query optimization strategies such as the ones described as a part of this article.
In our previous work atresigmod15 (), we had proposed to convert a BGPOPTUNION query into the UNF and process each of the subqueries in the UNF independently. This involves duplication of efforts to prune the triples associated with a triple pattern that appears in multiple UNF subqueries (due to rewrite rules). Through the method presented in this section, we avoid this by treating each unionfree BGPOPT component of the query independently, pruning it before bringing the query in UNF, and use normal sequence of multiwayjoin, followed by bestmatch wherever required by the structure of the query.
Lemma 9.1
Nullification and bestmatch can be avoided if each subquery in the Union Normal Form of a BGPOPTUNION query satisfies the following two conditions:

Considering only the GoSN of , GoT of triple patterns enclosed in each slave supernode of this GoSN is acyclic.

For every pair of masterslave supernodes such as in the GoSN of , there is only one equivalent class GoT edges.
9.2 FILTERs
SPARQL FILTER construct can have a complex Boolean expression that is supposed to be evaluated for every answer/result generated for the pattern over which it is applied. Below we have given an example of a BGPOPTFILTER query.
SELECT ?friend ?sitcom ?dir
\lastbox WHERE {
\lastbox :Jerry :hasFriend ?friend .
\lastbox ?friend :age ?age .
\lastbox ?friend :actedIn ?sitcom .
\lastbox OPTIONAL {
\lastbox ?sitcom :hasDirector ?dir .
\lastbox ?sitcom :location :NYC .
\lastbox }
\lastbox FILTER(?age 60 && ?dir != :Jerry)
\lastbox }
In essence, this query is asking for all the friends of :Jerry who have acted in a ?sitcom, and optionally information of the ?dir of the ?sitcom too, if it was located in :NYC. Notice that the FILTER condition further emphasizes that the ?friend must be younger than “60” years of age, and the ?sitcom’s director (?dir) cannot be :Jerry himself.
In general, FILTER expression of a SPARQL query can be notationally represented as , where is a SPARQL query pattern – BGP, OPTIONAL, UNION or any combination of these – and is a Boolean valued filter condition to be applied on . For the scope of our discussion, we assume safe filters perez2 (); polleres (), i.e., for , all the variables in appear in , . This is in line with our previous discussion of welldesigned OPTIONAL and UNION patterns. Unsafe (nonwelldesigned) filters can alter the semantics of the OPTIONAL patterns (ref perez2 () for more detailed discussion). The FILTER pattern evaluation techniques presented here can work with any combination of BGP, OPTIONAL, and FILTER, because BGPOPT remain as the building blocks of a SPARQL query, and our query evaluation techniques are built for these basic blocks.
For a SPARQL query with FILTERs, first we pushin the filter conditions as much as possible using the following rewrite rule on each UNIONfree subcomponent of the query without bringing the query in the UNF (the first three rewrite rules are described in Section 9.1).

.
This rule can be applied as our queries are assumed to be welldesigned. After pushingin the filters, we run prune_triples on each UNIONfree subcomponent of the query same as described in Section 9.1. We have an option to apply the FILTER conditions while loading the BitMat associated with each triple pattern. However, we can do so only if the respective FILTER expression satisfies following constraints.

FILTER expression is composed of conjunctive expression (only “” and no “”), and

each subexpression in the conjunction consists of only one variable (e.g., FILTER(?a 60 ?b 10)).
Then each subexpression of FILTER can be applied while loading the BitMat associated with the triple pattern that contains the respective variable in the FILTER subexpression. However if a FILTER expression consists of disjunction (“”), it can only be applied after multiwayjoin, once each result of the query is fully constructed. The decision of applying a filter during BitMat loading will also depend on the selectivity of the filter subexpression, and the available indexes. If these constraints are not satisfied, FILTER can be applied only in multiwayjoin. For simplicity, and the scope of this article, we assume that all the FILTER expressions in a BGPOPTUNIONFILTER query are applied only in the multiwayjoin procedure, as a part of the nullification process.
Note from Section 9.1, that if a query contains UNIONed patterns,
we first prune the triples in the UNIONfree subcomponents of the query using
prune_triples (Algorithm 1), then bring the query in the
UNF, and then evaluate each subquery independently using
multi
wayjoin. For a query with UNIONs and FILTER conditions, after
prune_triples, we bring the query in UNF by applying rewrite rules
1–3 described in Section 9.1, rule 4 described earlier in this
section, and additionally applying rule 5 below.

.
Any FILTER expressions that were not applied during BitMat loading are applied
as a part of the nullifi
cation procedure, when
multiwayjoin produces each result. Nullification not only
ensures consistent variable bindings (for any reordered inner and leftouter
joins), but also evaluates the filter conditions. If the filter condition fails
for the given pattern, we nullify all the variable bindings of that pattern and
any slaves of the pattern. Recall that here the GoSN of the query comes in handy
for quickly determining the nullification of any slave patterns. Once the
results of all the subqueries on UNF are generated, we union them all, and apply
bestmatch (minimum union) iff was rewritten as for some subquery, or at least one variable binding was
nullified during the nullification operation in
multiwayjoin, either as a result of a cyclic subquery or the filter
condition.
9.3 Distinct
A DISTINCT keyword in the SELECT clause eliminates duplicates from the results. Consider the following BGP query over an RDFized version of a movie database like IMDB, which is asking for all the distinct pairs of the actors () and their directors ().
SELECT DISTINCT ?a ?d
WHERE{
\lastbox ?m rdf:type :Movie .
\lastbox ?m :hasActor ?a .
\lastbox ?m :hasDirector ?d .}
:UmaThurman has acted in three movies directed by :QuentinTarantino, they are :PulpFiction, :KillBillVol1, and :KillBillVol2. Without the DISTINCT clause, this query will generate three copies of (:UmaThurman, :QuentinTarantino) as the variable bindings for (?a, ?d) in the results. With the DISTINCT clause we will get only one copy.
SPARQL algebra allows an arbitrary number of variables in the DISTINCT clause
(just like SQL). When there are multiple variables in the DISTINCT clause
(like in the example above), the distinct values are composite of the
bindings of the respective variables, e.g., (:UmaThurman,
:QuentinTarantino) is distinct from
(:UmaThurman,
:WoodyAllen), although they both share :UmaThurman. If the
variables in the DISTINCT clause appear in different triple patterns, like
in our example, we have to generate intermediate variable bindings of the other
variables not in the DISTINCT clause, and discard them later. This may create a
memory overhead. E.g., we first have to generate bindings of all three
variables (?m, ?a, ?d), project out only bindings of (?a, ?d),
and then pass these (?a, ?d) binding pairs through the DISTINCT filter
to remove duplicates. Hence for an arbitrary number of variables in the WHERE
and DISTINCT clauses, evaluation of a query may become memory intensive if the
DISTINCT clause is composed of many variables that do not appear in the same
triple pattern.
For a SPARQL query with any intermix of BGP, OPTIONAL, UNION, FILTER, we employ different techniques for the evaluation of the DISTINCT clause depending on the other clauses in the query. Since BGPOPT patterns are the basic building blocks of a SPARQL query, we begin with how to handle the DISTINCT clause with BGP and OPTIONAL patterns so as to avoid using extra memory overhead for the removal of nonessential variables, e.g., ?m in our example above.
9.3.1 Acyclic BGP
For a SPARQL query with just a basic graph pattern, i.e., innerjoins alone, if it is acyclic per the definition of acyclicity as presented in Section 3.1, we prune the triples using prune_triple (Algorithm 1), which leaves minimal triples in each BitMat associated with the triple patterns. Multwayjoin projects out all the variable bindings without duplicate removal. If we project out only some variables, we may get duplicates (in composite value form), as seen in the example elaborated at the beginning of this section. Hence we need a way to remove the bindings of nonessential variables, i.e., those not appearing in the DISTINCT clause, and remove duplicates from the composite bindings of the DISTINCT variables. For this discussion we assume the query to not have OPTIONAL, UNION, or FILTER patterns. Thus the GoSN of the query is going to have only one supernode, , and all the triple patterns are going to be encapsulated inside it. Considering the graph of triple patterns (GoT) with undirected edges between the triple patterns, we identify a Minimal Covering Subgraph (MCS) such that triple patterns in MCS cover all the variables appearing in the DISTINCT clause, and no other strict subgraph of this MCS covers DISTINCT variables – in short we identify the minimum triple patterns from GoT that are required to project out the DISTINCT variable bindings.
MCS can be identified methodically as follows. First identify all the triple patterns such that it has one or more DISTINCT variables in it, and then identify a minimal subgraph that connects all these triple patterns. Eliminate a from this subgraph, if it has at least one neighbor in this subgraph such that the , i.e., the DISTINCT variables appearing in also appear in , and elimination of does not disconnect the rest of the subgraph. We continue this process until we do not find any triple pattern to eliminate. This makes the minimal covering subgraph (MCS). This MCS is acyclic, because the original GoT from which the MCS is carved out is acyclic too. Thus conceptually this MCS represents a subquery of the original BGP query. Now we will operate on this MCS to eliminate nonessential variables – those not appearing in the DISTINCT clause but appearing in the MCS because they connect one or more DISTINCT variables. Before that we review some important properties of Boolean matrix multiplication (BMM) of BitMats. BitMats conceptually represent an RDF graph’s adjacency matrices.
Figure 9.2 shows the Basic Graph Pattern (BGP) and the graph of triple patterns (GoT) of the query given at the beginning of this section. Triple patterns ?m :hasActor ?a and ?m :hasDirector ?d are connected to each other with label ?m in the GoT. Note that these two triple patterns are part of the MCS due to ?a and ?d variables, but the triple pattern ?m rdf:type :Movie is not. The BitMats associated with these triple patterns contain all the triples which have predicates (edgelabels) :hasActor and :hasDirector respectively, and they in turn represent the adjacency matrices of two subgraphs of the original RDF graph, with only :hasActor and :hasDirector edge labels respectively. The transpose of the BitMat of :hasDirector conceptually reverses the direction of edges between the respective RDF nodes and represents a triple pattern ?d :hasDirector ?m. Thus if we do a BMM of the transpose of BitMat of :hasDirector with the BitMat of :hasActor, the resultant matrix represents all the distinct pairs of nodes that have at least one undirected path with edge labels :hasDirector–:hasActor between them, and eliminates the bindings of ?m. It, in turn, eliminates the RDF nodes representing ?m, which is a common variable between the two triple patterns. Figure 9.3 pictorially represents this concept of Boolean Matrix Multiplication. Note that since this query is acyclic, and we have already done prune_triple, and have minimal triples left in it, thus their BMM gives us the exact distinct pairs of in .
Property 9.1
An edge between two triple patterns in a minimal covering subgraph signifies a potential Boolean matrix multiplication between them to eliminate the variables common between them, which appear as the edge label.
We make use of this property to methodically eliminate nonessential variables in the MCS to shrink it further, and get a set of BitMats absolutely required to project the DISTINCT variable bindings. The MCS shrinking process is carried out as follows.

For every pair of triple patterns and that have an undirected edge between them with label, say ?m, such that is a nonessential variable, do a BMM of and such that bindings for ?m get eliminated. Whether we need to take a transpose of the BitMat or not depends on whether we have loaded SO or OS BitMat for the respective triple pattern (ref Section 6). E.g., in our example if we have loaded OS BitMat (?m :hasDirector ?d) and SO BitMat of (?m :hasActor ?a), then we do a BMM of with no change in the respective BitMats. However if we have loaded SO BitMat of (?m :hasDirector ?d) or OS BitMat of (?m :hasActor ?a), we take the transpose of them before doing the BMM. Let us denote the resultant BitMat as .

can connect to any neighbor of or (excluding and ), if it shares the exact same variables with the respective neighbor as done by or . If can thus connect to all of and ’s neighbors, we remove both and by connecting to their neighbors.

may not be able to connect to all of the and ’s neighbors, if or connects with other triple patterns over the same edge label between and , e.g., . In that case we keep either of or , eliminate the other. We connect to the preserved triple pattern and eliminated triple pattern’s neighbors (whichever it can connect to). This procedure of preserving either of or is explained further. If is connected to over, say , cannot connect to any neighbors of and that are connected over . By preserving either of or , we ensure that the remaining MCS remains connected.

We continue this process of eliminating triple patterns and their respective BitMats, until we have an MCS where each edge label contains a DISTINCT variable, or it has only one BitMat.
In Figure 9.4, we have shown a sample query where we need to eliminate the bindings of , while preserving correlations between the bindings of , , , . The figure also shows an evolution of this MCS to eliminate using the above algorithm. Intuitively, we eliminate all the intermediate nonrequired variables, by establishing a direct correlations between the bindings of the required variables. E.g., when and are part of two different triple patterns connected over , bindings of and are correlated through . When we do a BMM, = , we establish a direct correlation between the bindings of the pair, and eliminate the need of having as an intermediary.
This algorithm is monotonic – at the end of one iteration, the edges, nodes, and BitMats in an MCS remain the same or become fewer than before. It gradually eliminates the edges with join variables that do not appear in the DISTINCT clause. At the end of the procedure, we are left with an MCS with BitMats – either original or new ones created in the process of BMM – and edges with only DISTINCT variables. This algorithm always converges when all the nonrequired variables are eliminated from the MCS. Also note that the total BitMats at the end of the algorithm are always fewer than the original BitMats in the query – note that in step 2, we remove two BitMats while creating a new one, and in step 3 we remove one BitMat and create one. Hence eventually we are left with fewer BitMats – thus reducing the memory requirements.
9.3.2 Acyclic BGPOPT queries
For the queries with an intermix of BGP and OPTIONAL patterns, we consider queries whose (a) GoT is completely acyclic, and (b) queries with cyclic GoT.
Like an acyclic BGP query, for an acyclic BGPOPT query, prune_triples ensures minimal triples associated with each triple pattern in the query. We eliminate nonessential variables before doing multiwayjoin as follows. Starting with , we go over all the supernodes in the masterslave hierarchy (masters before their respective slaves). We mark a supernode if it contains one or more variables appearing in the DISTINCT clause that do not appear in any of its masters. This is because a variable’s binding from the master triple pattern always dominates the binding from a slave triple pattern. So there is no need to consider a slave supernode if all of its DISTINCT variables are covered by one or more of its master. After this, we carve out a Minimal Covering GoSN (MCGoSN) such that all the marked nodes are in a minimal connected subgraph of GoSN.
An MCGoSN is connected if there there is an undirected path from one supernode to another disregarding the directionality of the edges connecting the supernodes (recall our GoSN construction from Section 2). Since we have a unique in our GoSN, any connected MCGoSN always contains . Now there are two cases – (1) and every supernode in MCGoSN contains at least one DISTINCT variable, (2) all the DISTINCT variables are contained in one or more slaves. For the second case, we cannot use the Boolean Matrix Multiplication technique to eliminate nonessential variables, and thus for this type of query we have to resort to standard way of listing all the bindings of DISTINCT variables, and then removing duplicates.
For the first case however, we can use the BMM technique as follows. We order all the supernodes in MCGoSN per masterslave hierarchy. We start with and the triple patterns in it, and carve out a Minimal Covering Subgraph (MCS) from the GoT of only these triple patterns that cover the DISTINCT variables that appear in . Next we iteratively go over the next supernode in the masterslave order, say . We extend the previously carved MCS to minimally cover the triple patterns in for the DISTINCT variables that appear only in , but not in any of its masters. At the end of this exercise we get an MCS that contains triple patterns from different supernodes.
Next, we need to eliminate the nonessential join variables and triple patterns from this MCS in the same way we did for pure BGP queries. However, in this MCS, we may have a masterslave hierarchy between the two neighboring triple patterns. We take care of this hierarchy as follows. Pairs of connected triple patterns, can be categorized as – (a) both and , (b) is a master of or vice versa, (c) and are slave peers contained in the same slave supernode, (d) and are contained in different slave supernodes.
Starting with type (a) pairs first, we completely reduce the part of MCS contained in . This reduction will cause some changes in the MCS nodes and connections. Next we consider all (b) type pair of nodes in MCS. Let us assume is the master of . If the edgelabel between is a nonessential variable, we do a BMM, remove , and reconnect to the newly created BitMat. Once we are done with all type (a) and (b) pairs, we shrink (c) type pairs same as the acyclic BGP technique. We ignore the (d) type edges. That is because recall that we assume OPTIONAL pattern queries to be welldesigned, so even if there are (d) type edges in the MCS, and always have an indirect path connecting them through their masters. Thus we gradually shrink this MCS to leave only the BitMats containing the bindings of the DISTINCT variables. Then we run multiwayjoin on these BitMats in the standard way in the masterslave hierarchical order.
9.3.3 Cyclic BGPOPT queries
For the cyclic BGPOPT queries, the minimality of triples after prune_triples cannot be guaranteed, so we cannot use this technique. For such queries, we have to enumerate the results with duplicates, and then remove of them using a naïve method.
9.3.4 Union, Filter
For queries with UNION or FILTER clauses along with BGP and OPT patterns, we cannot use the optimization technique of carving out an MCS from the entire query. The reason is – we can carve out an MCS for DISTINCT variable binding projections only if each triple pattern in the query has minimal triples associated with it. In a BGPOPT query with UNION pattern, we do the pruning of UNIONfree BGPOPT subpatterns in the given query (as described in Section 9.1). Then get the query in the UNF , and perform multiwayjoins on each in the UNF. However, since we did the pruning only on the UNIONfree BGPOPT subparts of the original query, the triples associated with the each triple pattern in may not be minimal.
The method presented in Section 9.2 for handling FILTERs in DISTINCTfree queries shows that an arbitrary FILTER condition can cause nullification while doing multiwayjoins, thereby altering the minimality of triples associated with the triple patterns onthefly. Hence for the SPARQL queries asking for DISTINCT projection of some variables, where the query body has UNIONs or FILTER conditions, we use the naïve way of projecting out all the variable bindings of the DISTINCT variables, and then sorting to remove any duplicates.
10 Related Work
SPARQL BGP queries are similar to SQL innerjoins, and thus naturally a lot of SQL innerjoin optimization techniques have been applied to SPARQL BGP query optimization. While the BGP query optimization has got a lot of attention, the discussion about optimization of other SPARQL components such as OPTIONAL patterns, UNIONs, FILTERs, DISTINCT clauses is quite sparse. This is despite the fact that in the context of SPARQL queries, these other components do make as large as 94% of the queries usewod11 (); swim (); manvmachine (); practsparql (). Previous work arenas (); iswc14 (); perez2 (); schmidt (), has extensively analyzed the semantics of welldesigned OPTIONAL patterns from the perspective of tractability properties. However, these texts have not focused on the discussion of other SPARQL components that are covered in this article. The idea of query graph of supernodes (GoSN) presented in this paper is reminiscent of welldesigned pattern trees (WDPT) perez (); letelier2 (), but WDPTs are undirected and unordered, whereas GoSN is directed, and establishes an order among the patterns (masterslave, peers), which is an integral part of our optimization techniques. Also while previous discussion has focused on WDPTs and tractability results, they have not taken into consideration the aspects of optimization techniques from the point of view of minimality of triples, and the order of processing semijoins and multiwayjoins, which make the performance intensive components of query evaluation techniques.
GalindoLegaria, Rosenthal galindosigmod (); galindolegaria1 (); galindolegaria2 () and Rao et al rao2 (); rao1 () have proposed ways of achieving SQL leftouterjoin optimization through reordering inner and leftouter joins. Their work is closest to the work in this article. Rao et al have proposed nullification and bestmatch operators to handle inconsistent variable bindings and subsumed results respectively (see Section 4). In their technique, nullification and bestmatch are required for each reordered query, as the minimality of tuples is not guaranteed. They do not use methods like prune_triples to eliminate unwanted tuples before joins. Bernstein et al and Ullman semij2 (); semij1 (); ullman () have proved the properties of minimality for acyclic innerjoins only. Through our work, we have taken a major step forward by extending these properties in the context of SPARQL OPTIONAL patterns (SQL leftouterjoins), and finding ways to avoid overheads like nullification and bestmatch operations. We have also extended our previous results presented in atresigmod15 () about the class of queries that can avoid nullification and bestmatch despite reordering of inner and leftouter joins.
Additionally, in this article we have extensively analyzed the UNION, FILTER, and DISTINCT clauses of SPARQL from the point of view of minimality of triples, ways of unioning the results (minimumunion versus standard unionall), and structural aspects of queries (cyclicity). With this analysis we have shown that a large number of – acyclic as well as some (good) cyclic – SPARQL queries can be optimized by using our techniques of BGPOPT query processing as building blocks, and can avoid the additional overheads of processing and indexing for the correctness of the results. Our article also throws a new light on the treatment of NULL values in the RDF and SPARQL context, and the implications of various SPARQL components – other than Basic Graph Patterns (innerjoins) – in the presence of NULL values in the query results. To the best of our knowledge, this topic, which directly affects implementation and optimization methods, has not been handled before.
For innerjoin optimization, RDF engines like TriAD triad (), RDF3X rdf3x (), gStore gstore () take the approaches like graph summarization, sidewaysinformationpassing etc for an early pruning of triples. Systems like TripleBit triplebit () use a variable length bitwise encoding of RDF triples, and a query plan generation that favors queries with “star” joins, i.e., many triple patterns joining over a single variable. RDF engines built on top of commercial databases such as DB2RDF ibmsigmod13 () propose creation of entityoriented flexible schemas and better dataflow techniques through the query plan to improve the performance of “star” join queries. Along with this, there are distributed RDF processing engines such as HRDF3X hrdf3x () and SHARD shard ().
While many of these engines mainly focus on efficient indexing of RDF graphs, BGP queries (innerjoins), and exploiting “star” patterns in the queries, we have focused on the broader components of SPARQL patterns such as OPTIONALs, UNIONs, FILTERs, and DISTINCT, which cannot always exploit the benefits of innerjoin focused query optimizers.
11 Conclusion
In this article we have done an extensive analysis of a wide range of SPARQL components such as OPTIONAL patterns, UNIONs, FILTERs, DISTINCTs, and proposed that they can be evaluated by simply using our BGPOPT pattern’s optimization techniques as building blocks. We have extended the previously proposed concepts of minimality of triples (tuples), cyclicity of queries, and nullification, bestmatch operations. With this first of a kind analysis of a wide range of SPARQL components, we hope to create novel optimization techniques for performance intensive SPARQL components. Our analysis shows that this is be possible by simple semantic manipulation of various intermixed query components. Our article also elaborately discusses the treatment of NULL values in the RDF and SPARQL context.
Since there is a very close resemblance between SPARQL components and SQL queries, our proposed techniques, as well as observations about query’s structural properties and optimization opportunities, can be directly applicable to the respective SQL components too.
References
 (1) BitMat sources. http://sourceforge.net/projects/bitmatrdf/.
 (2) RDF 1.1 Ntriples. http://www.w3.org/TR/ntriples/.
 (3) SPARQL 1.1 Query Language. http://www.w3.org/TR/2013/RECsparql11query20130321/.
 (4) M. Arenas and J. Pérez. Querying semantic web data with SPARQL. In PODS, 2011.
 (5) M. Atre. Left Bit Right: For SPARQL Join Queries with OPTIONAL Patterns (Leftouterjoins). In SIGMOD, 2015.
 (6) M. Atre, V. Chaoji, M. J. Zaki, and J. A. Hendler. Matrix “Bit”loaded: A Scalable Lightweight Join Query Processor for RDF Data. In WWW, 2010.
 (7) P. A. Bernstein and D. W. Chiu. Using semijoins to solve relational queries. Journal of the ACM, 28(1), 1981.
 (8) P. A. Bernstein and N. Goodman. Power of natural semijoins. SIAM Journal of Computing, 10(4), 1981.
 (9) M. A. Bornea, J. Dolby, A. Kementsietsidis, K. Srinivas, P. Dantressangle, O. Udrea, and B. Bhattacharjee. Building an Efficient RDF Store over a Relational Database. In SIGMOD, 2013.
 (10) Egor V. Kostylev and Bernardo Cuenca Grau. On the Semantics of SPARQL Queries with Optional Matching under Entailment Regimes. In ISWC, 2014.
 (11) C. A. GalindoLegaria. Outerjoins As Disjunctions. In SIGMOD, 1994.
 (12) C. A. GalindoLegaria and A. Rosenthal. How to Extend a Conventional Optimizer to Handle One and TwoSided Outerjoin. In ICDE, 1992.
 (13) C. A. GalindoLegaria and A. Rosenthal. Outerjoin Simplification and Reordering for Query Optimization. ACM Trans. on Database Systems, 22(1), 1997.
 (14) M. A. Gallego, J. D. Fenández, M. A. MartínezPrieto, and P. de la Fuente. An Empirical Study of RealWorld SPARQL Queries. In Usage Analysis and the Web of Data at WWW, 2011.
 (15) S. Gurajada, S. Seufert, I. Miliaraki, and M. Theobald. TriAD: A Distributed Sharednothing RDF Engine Based on Asynchronous Message Passing. In SIGMOD, 2014.
 (16) X. Han, Z. Feng, X. Zhang, X. Wang, G. Rao, and S. Jiang. On the Statistical Analysis of Practical SPARQL Queries. In WebDB, 2016.
 (17) J. Huang, D. J. Abadi, and K. Ren. Scalable SPARQL querying of large RDF graphs. PVLDB, 2011.
 (18) A. Letelier, J. Pérez, R. Pichler, and S. Skritek. Static Analysis and Optimization of Semantic Web Queries. In PODS, 2012.
 (19) A. Letelier, J. Pérez, R. Pichler, and S. Skritek. Static Analysis and Optimization of Semantic Web Queries. ACM Trans. on Database Systems, 38(4), 2013.
 (20) T. Neumann and G. Weikum. Scalable join processing on very large RDF graphs. In SIGMOD, 2009.
 (21) J. Pérez, M. Arenas, and C. Gutierrez. Semantics and complexity of SPARQL. ACM Trans. on Database Systems, 34(3), 2009.
 (22) F. Picalausa and S. Vansummeren. What are real SPARQL queries like? In SWIM workshop at SIGMOD, 2011.
 (23) A. Polleres. From SPARQL to Rules (and back). In WWW, 2007.
 (24) J. Rao, B. G. Lindsay, G. M. Lohman, H. Pirahesh, and D. E. Simmen. Using EELs, a Practical Approach to Outerjoin and Antijoin Reordering. In ICDE, 2001.
 (25) J. Rao, H. Pirahesh, and C. Zuzarte. Canonical Abstraction for Outerjoin Optimization. In SIGMOD, 2004.
 (26) L. Rietveld and R. Hoekstra. Man vs. Machine Differences in SPARQL Queries. In USEWOD workshop at ESWC, 2014.
 (27) K. Rohloff and R. E. Schantz. Clauseiteration with MapReduce to Scalably Query Datagraphs in the SHARD Graphstore. In DIDC, 2011.
 (28) M. Schmidt, M. Meier, and G. Lausen. Foundations of SPARQL Query Optimization. In ICDT, 2010.
 (29) J. D. Ullman. Principles of Database and KnowledgeBase Systems, Volume II. Computer Science Press, 1989.
 (30) P. Yuan, P. Liu, B. Wu, H. Jin, W. Zhang, and L. Liu. TripleBit: A Fast and Compact System for Large Scale RDF Data. In PVLDB, 2013.
 (31) L. Zou, J. Mo, L. Chen, M. T. Özsu, and D. Zhao. gStore: Answering SPARQL Queries via Subgraph Matching. In PVLDB, 2011.