2.1 Incomplete databases and constraints

We represent missing information in relational databases in the standard way using nulls (Abiteboul et al. Reference Abiteboul, Hull and Vianu1995; Imielinski and Lipski Reference Imielinski and Lipski1984; van der Meyden Reference van der Meyden, Chomicki and Saake1998). Incomplete databases are populated by constants and nulls, coming respectively from two countably infinite sets ${\sf Const}$ and ${\sf Null}$ . We denote nulls by $\bot$ , sometimes with sub- or superscript. We also allow them to repeat, thus adopting the model of marked nulls, as customary in the context of applications such as OBDA or data integration and exchange.

A relational schema, or vocabulary $\sigma$ , is a set of relation names with associated arities. A database D over $\sigma$ associates to each relation name of arity k in $\sigma$ , a k-ary relation which is a finite subset of $({\sf Const} \cup {\sf Null})^k$ . Sets of constants and nulls occurring in D are denoted by ${\sf Const}(D)$ and ${\sf Null}(D)$ . A database is complete if it contains no nulls, that is ${\sf Null}(D) = \emptyset$ .

The active domain of D is the set of all values appearing in D, that is $\text{adom}(D)={\sf Const}(D)\cup{\sf Null}(D)$ .

A valuation $v : {\sf Null}(D) \rightarrow {\sf Const}$ on a database D is a map that assigns constant values to nulls occurring in D. By v(D) and $v(\bar a),$ we denote the result of replacing each null $\bot$ by $v(\bot)$ in a database D or in a tuple $\bar a$ . The semantics $[\![ D ]\!]$ of an incomplete database D is the set of all complete databases it can represent. Here as is common in research on incomplete data, we use closed-world assumption (Imielinski and Lipski Reference Imielinski and Lipski1984; Reiter Reference Reiter, Gallaire and Minker1977) (i.e. everything we do not know to be true is automatically assumed to be false and no new tuple can be added).

An equality-generating dependency (EGD) is a first-order sentence of the form $\forall \bar x ~~(\varphi(\bar x)~\rightarrow~z=z')$ , where $\varphi(\bar x)$ is a conjunction of atoms (without constants), each variable in $\bar x$ occurs in some atom of $\varphi$ , and z, z’ are distinct variables in $\bar x$ . As a special case, a functional dependency (FD) over a relation name R is of the form $\forall \bar x, \bar y, z, z'~~(R(\bar x, \bar y, z) \wedge R(\bar x, \bar y', z') \rightarrow z = z')$ . Throughout this paper, we will assume that a (possibly empty) set of EGDs $\Sigma$ is associated with the database schema $\sigma$ .

A valuation v is consistent with $\Sigma$ (or just consistent, when $\Sigma$ is clear from the context) if $v(D) \models \Sigma$ . We denote by the set of all consistent valuations defined on D.

2.2 Query answering

An m-ary query Q of active domain $\text{adom}(Q) \subseteq {\sf Const}$ is a map that associates with a database D a subset of $(\text{adom}(D)\cup \text{adom}(Q))^m$ . To answer an m-ary query Q over an incomplete database D, we follow (Lipski Reference Lipski, Rosenkrantz and Fagin1984) and adopt a slight generalization of the usual intersection-based certain answers notion, defined as $\cap_vQ(v(D))$ , and furthermore incorporate constraints into query answering.

The set of certain answers to Q over D, with respect to a set of constraints $\Sigma$ , is

For queries that explicitly use constants, we shall expand this to allow $\bar a$ range over $\text{adom}(D)$ and those constants. The only difference with the usual notion is that we allow answers to contain nulls, to avoid pathological situations when answers known with certainty are not returned (e.g. in a query returning a relation R one would expect R to be returned while the intersection-based certain answer will only return null-free tuples). If the set of constraints $\Sigma$ is empty, we omit it and write simply $\mathsf{cert}(Q,D)$ . Of course, every valuation is consistent with the empty set of constraints.

Following Libkin (Reference Libkin, den Bussche and Arenas2018), given a query Q, a database D, a set of constraints $\Sigma$ , and a tuple $\bar a$ over $\text{adom}(D)\cup \text{adom}(Q)$ , we let the support of $\bar a$ be the set of all valuations that witness it:

Again if $\Sigma=\emptyset$ we omit the subscript.

Supports thus measure how close a tuple is to certainty. We consider one answer to be better than another if it has more support. That is, given a database D, a k-ary query Q, and k-tuples $\bar a, \bar b$ over $\text{adom}(D)\cup\text{adom}(Q)$ , we let

The set of best answers to Q over D is defined as the set of answers for which there is no better one:

As the set of certain answers to Q over D is the set of answers that are witnessed by all valuations, note that it could also be defined using the notion of support. Namely, $\mathsf{cert}_\Sigma(Q,D)$ consists of all tuples $\bar{a} \in \text{adom}(D)^m$ for which contains all consistent valuations in.

We have, and.

It follows that $\mathsf{cert}(Q,D) = \emptyset$ and.

2.3 Naïve evaluation and certain answers

For a query Q written in FO or Datalog, we write Q(D) to mean that such a query is evaluated na ve ly. That is, if D contains nulls, nulls of D are treated as new constants in the domain of D, distinct from each other, and distinct from all the other constants in D and $\varphi$ . For example, the query $\varphi(x,y) = \exists z~(R(x, z) \wedge R(z, y))$ , on the database $D = \{R(1, \bot_1), R(\bot_1, \bot_2), R(\bot_3, 2)\}$ selects only the tuple $(1, \bot_2)$ .

There are known connections between na ve evaluation and certain answers. If $\Sigma$ is empty and Q is a union of conjunctive queries, then $\mathsf{cert}_\Sigma(Q,D)=Q(D)$ , see Imielinski and Lipski (Reference Imielinski and Lipski1984). If $\Sigma$ contains a set of EGDs, then $\mathsf{cert}_\Sigma(Q,D)=Q\big(\mathsf{chase}_\Sigma(D)\big)$ ; cf. Greco et al. (Reference Greco, Molinaro and Spezzano2012). Here $\mathsf{chase}_\Sigma$ refers to the standard chase procedure with a set of EGDs, see Abiteboul et al. (Reference Abiteboul, Hull and Vianu1995).

2.4 Query languages

Here we shall study best and certain answers to first-order ( ${{{{\mathsf{FO}}}}}$ ) queries, possibly in the presence of constraints, by means of their rewriting in ${{{{\mathsf{FO}}}}}$ and Datalog. FO queries of vocabulary $\sigma$ use atomic relational and equality formulae and are closed under Boolean connectives $\wedge, \vee, \neg$ and quantifiers $\exists, \forall$ . We write $\varphi(\bar x)$ for an ${{{{\mathsf{FO}}}}}$ -formula $\varphi$ with free variables $\bar x$ . With slight abuse of notation, $\bar x$ will denote both a tuple of variables and the set of variables occurring in it. The set of constants used by $\varphi$ is denoted by $\text{adom}(\varphi)$ . We interpret ${{{{\mathsf{FO}}}}}$ -formulas under active domain semantics, that is quantified variables range over $\text{adom}(D) \cup \text{adom}(\varphi)$ . Thus, an ${{{{\mathsf{FO}}}}}$ -formula $\varphi(\bar x)$ represents a query (of active domain $\text{adom}(\varphi)$ ) mapping each database D into the set of tuples $\{\bar t \textrm { over } \text{adom}(D)\cup\text{adom}(\varphi)~|~D \models \varphi(\bar t)\}$ .

To evaluate ${{{{\mathsf{FO}}}}}$ -formulas with free variables, we use assignments $\nu$ from variables to constants in the active domain. Note that with a little abuse of notation, we write $D \models \varphi(\bar t)$ for $D \models_\nu \varphi(\bar x)$ under the assignment $\nu$ sending $\bar x$ to $\bar t$ .

Here it is important to note that the query associated with $\varphi$ is a mapping defined on all databases D, possibly with nulls. If D contains nulls, $D \models \varphi(\bar t)$ is to be intended “navely,” that is nulls of D are treated as new constants in the domain of D, distinct from each other, and distinct from all the other constants in D and $\varphi$ . For example, the query $\varphi(x,y) = \exists z~(R(x, z) \wedge R(z, y))$ , on the database $D = \{R(1, \bot_1), R(\bot_1, \bot_2), R(\bot_3, 2)\}$ selects only the tuple $(1, \bot_2)$ .

Conjunctive queries (CQs) are given by the $\exists,\wedge$ -fragment of ${{{{\mathsf{FO}}}}}$ , and their unions (UCQs) by the $\exists,\wedge,\vee$ -fragment of ${{{{\mathsf{FO}}}}}$ ; these are also captured by the positive fragment of relational algebra (select-project-union-join queries).

We also consider Boolean combination of conjunctive queries (BCCQs), that is, the closure of conjunctive queries under operations $q \cap q'$ , $q \cup q'$ , and $q - q'$ .

A Datalog rule (Abiteboul et al. Reference Abiteboul, Hull and Vianu1995) is an expression of the form $R_1(u_1) \leftarrow R_2(u_2), \ldots , R_n (u_n )$ where $n \geq 1$ , $R_1, \ldots , R_n$ are relation names and $u_1,\ldots, u_n$ are tuples of appropriate arities. Each variable occurring in $u_1$ must occur in at least one of $u_2, \ldots , u_n$ . A Datalog program is a finite set of Datalog rules. The head of the rule is the expression $R_1(u_1)$ ; and $R_2(u_2), \ldots, R_n(u_n )$ forms the body. The semantics is the standard fixed-point semantics.

As the language of our rewritings, we shall be using ${{{{\mathsf{FO}}}}}$ , but also a fragment of stratified Datalog with negation in bodies that can be seen in two different ways.

1. 1. A program is evaluated in two steps. First, we can have a Datalog program P defining new idb predicates $S_1,\ldots,S_\ell$ . Then, we ask an ${{{{\mathsf{FO}}}}}$ query over the schema extended with these predicates $S_1,\ldots,S_\ell$ .

2. 2. We evaluate a stratified Datalog with negation program in which the first stratum has no negation (but may have recursion) and the second stratum has no recursion (but may have negation).

From the rewritings, we produce it will be clear that they fall in these classes. The key point about them is that they can be implemented in recursive SQL and that they both have PTIME data complexity, making their evaluation feasible. Note that recursive SQL as it is currently implemented, for example, in PostgreSQL 8.4, is actually Turing complete (Gierth Reference Gierth2011; Coelho Reference Coelho2013).

2.5 Complexity classes

In order to study the complexity of best and certain answer computation, we shall need two classes in the second level of the polynomial hierarchy. Both of these contain NP and coNP and are contained in $\Sigma^p_2\cap\Pi^p_2$ . The class DP consists of languages $L_1\cap L_2$ where and. This class has appeared in database applications (Fagin et al. Reference Fagin, Kolaitis and Popa2005; Barceló et al. Reference Barceló, Libkin and Romero2014). The class consists of problems that can be solved in polynomial time with a logarithmic number of calls to an NP oracle (Buss and Hay Reference Buss and Hay1991). Equivalently, it can be described as the class of problems solved in P with an NP oracle where calls to the oracle are done in parallel, that is, independent of each other. This class has appeared in the context of AI, modal logic, OBDA (Gottlob Reference Gottlob1995; Eiter and Gottlob Reference Eiter and Gottlob1997; Calvanese et al. Reference Calvanese, Giacomo, Lembo, Lenzerini and Rosati2006; Bienvenu and Bourgaux Reference Bienvenu and Bourgaux2016), data exchange (Arenas et al. Reference Arenas, Pérez and Reutter2013).

4.1 A normal form for queries: neutralizing variable repetition

Towards our rewritings, we start by putting each conjunctive query in a normal form which eliminates repetition of variables, by introducing new equality atoms.

We say that $q(\bar{w})$ is the relational subquery of Q, and $e(\bar{x}, \bar{w})$ is the equality subquery of Q. A BCCQ is in NRV normal form if it is a Boolean combination of CQs in NRV normal form.

Clearly every conjunctive query Q is equivalent to a query in NRV normal form; moreover, Q can be easily rewritten in NRV normal form (in linear time in the size of the query). Thus in what follows, we assume w.l.o.g. that conjunctive queries are given in NRV normal form. Intuitively, the NRV normal form allows us to separate the two ingredients of a conjunctive query: the existence of facts in some relations of the database, on the one side, and a set of equality conditions on data values occurring in these facts, on the other side. The existence of facts does not depend on the valuation of nulls and thus can be directly tested on the incomplete database. Instead equality atoms in an NRV normal form imply conditions that valuations need to satisfy in order for the query to hold. We can thus first concentrate on the support of equality subqueries. This will be encoded in ${{{{\mathsf{FO}}}}}$ and then integrated in the rewriting of the whole conjunctive query.

We introduce a notion of equivalence of database elements w.r.t. to a set of equalities. Intuitively, equivalent elements of a tuple $\bar t$ are the ones which should be collapsed into a single value in order for a valuation of $\bar t$ to satisfy all the given equalities.

Definition 4.3 Given a database D, a conjunction of equality atoms $\gamma(\bar{y})$ and an assignment $\nu: \bar y \cup \text{adom}(\gamma) \rightarrow\text{adom}(D)\cup \text{adom}(\gamma)$ preserving constants, we say that $u,u' \in \text{adom}(D)\cup\text{adom}(\gamma)$ are equivalent w.r.t. $\gamma$ and $\nu$ and write $u \equiv_\gamma^\nu u'$ , if either $u=u'$ or (u, u’) belongs to the reflexive symmetric transitive closure of $\{(\nu(x), \nu(w)) ~|~ x = w \in \gamma\}$ .

The relation $\equiv_\gamma^\nu$ is clearly an equivalence relation over $\text{adom}(D) \cup\text{adom}(\gamma)$ , where each element outside the range of $\nu$ forms a singleton equivalence class.

In what follows, we denote by $\sim_{\gamma}$ the reflexive symmetric transitive closure of $\{ (x,w) ~|~ x = w \in \gamma \}$ . Note that this is an equivalence relation among variables and constants of $\gamma$ . We will provide two syntactic encodings of this relation, one in Datalog and one in FO.

4.2 Datalog rewriting for certain answers for BCCQs with EGDs

Recall that, given a query Q, a database D, and a tuple $\bar{a}$ over $\text{adom}(D) \cup \text{adom}(Q)$ we let the support of $\bar{a}$ be the set of all valuations that witness it:

In order to look for rewritings of BCCQs, a key observation is that $\bar a$ is a certain answer to Q iff. When Q is a BCCQ, so is $\neg Q$ , thus we look for ways of expressing (non-)emptiness of the support for BCCQs.

We start by concentrating on the support of equality subqueries. This will be encoded in Datalog and then integrated, as a key ingredient, in the rewriting of the whole query. We let $\gamma(\bar y)$ be an arbitrary set of equality atoms among variables $\bar y$ and possibly constants. Intuitively, we will be interested in the case that $\gamma (\bar y)$ is the equality subquery $e(\bar x, \bar w)$ of a CQ in NRV normal form (thus notice that in the Datalog program below $\bar y$ encompasses variables $\bar x \bar w $ of an equality subquery).

Remark that we can always write an EGD so that no variable in its body occurs more than once; it suffices to add to the body a set of variable equalities. Thus, we assume that EGDs in $\Sigma$ are of the form $\forall \bar u ((\varphi(\bar{u}) \wedge \psi) \rightarrow z=z')$ where z, z’ are in $\bar u$ , the conjunction of atoms $\varphi(\bar u)$ contains no constants, no variable occurs twice in $\varphi(\bar u)$ , and $\psi$ is a set of equalities among variables of $\bar u$ . Remark also that membership in the set $\text{adom}(D) \cup\text{adom}(\gamma)$ can be expressed by a UCQ formula that we call Dom(x). We encode equivalence of database elements in $\text{adom}(D) \cup\text{adom}(\gamma)$ w.r.t. a set of equalities $\gamma(\bar y)$ using the following Datalog program ^{Footnote 1} :

Intuitively, if $\bar t$ is a tuple of database elements assigned to $\bar y$ , equivalent elements of D are the ones which should be collapsed into a single value in order for a valuation of D to satisfy all the equalities $\gamma (\bar t)$ and the EGDs. For fixed $\gamma$ and $\bar t$ , the relation $\{(s,s') ~|~ D \models equiv_{\gamma}(\bar t, s, s')\}$ is an equivalence relation over $\text{adom}(D) \cup \text{adom}(\gamma)$ where each element of $\text{adom}(D)$ neither in $\bar t$ nor in $\text{adom}(\gamma)$ forms a singleton equivalence class.

The formula $equiv_\gamma$ is a key ingredient in our rewriting; as formalized in the following lemma, it selects precisely the pairs of elements that a consistent valuation needs to collapse to satisfy a set of equalities.

$\Leftarrow$ Assume $\forall s, s'$ , $D \models equiv_{\gamma}(\bar{t},s,s')$ implies $v(s)=v(s')$ . We show that $v(D) \models \gamma(v(\bar{t}))$ . In fact for each $(y_k = y_l) \in \gamma$ , we have $D \models equiv_\gamma(\bar t, t_k, t_l)$ (derived by the second rule). By our hypothesis $v(t_k) = v(t_l)$ , thus $\gamma(v(\bar t))$ holds.

Formulas we write in the remainder are over signature $\sigma\cup Null$ , where $\sigma$ is the database schema. In any incomplete database D over $\sigma \cup Null$ , Null is always interpreted by the set of nulls occurring in D (in accordance with the semantics of the SQL construct IS NULL). That is we allow rewritings to test whether a database element is null or not.

For $\gamma(\bar y)$ a conjunction of equality atoms, using $equiv_\gamma$ we define a new formula $comp_\gamma(\bar{y})$ stating the existence of a consistent valuation that collapses all equivalent elements of a tuple:

$$comp_{\gamma}(\bar{y})~:=~\forall z z'(equiv_{\gamma}(\bar{y},z,z') \wedge \neg Null(z) \wedge \neg Null(z')\rightarrow z=z')$$

In fact, consider an arbitrary EGD $(\varphi(\bar{u})\wedge \psi(\bar u)) \rightarrow z=z')$ in $\Sigma$ and assume $v(D) \models \varphi(\mu(\bar{u}))\wedge \psi(\mu(\bar u))$ for some $\mu$ ; we prove $\mu(z) = \mu(z')$ . Since $\varphi$ is a conjunction of atoms with no constants and no repeated variables, there exists $\mu'$ such that $D \models \varphi(\mu'(\bar u))$ and $v(\mu'(\bar u)) = \mu(\bar u)$ . Take any equality $w = w'$ in $\psi(\bar u)$ , we show that $D \models equiv_\gamma(\bar t, \mu'(w), \mu'(w'))$ . In fact because $v(D) \models \psi(\mu(\bar u))$ we have $\mu(w) = \mu(w')$ , and therefore $v(\mu'(w)) = v(\mu'(w'))$ . By definition of v then $D \models equiv_\gamma(\bar t, \mu'(w),\mu'(w'))$ . By the last rule of the Datalog program defining $equiv_\gamma$ , we thus have that $D \models equiv_\gamma(\bar t, \mu'(z), \mu'(z'))$ ; then again by definition of v, we have $v(\mu'(z))=v(\mu'(z')$ , and therefore $\mu(z) = \mu(z')$ . This shows that v(D) satisfies all the EGDs; thus, v is consistent.

We have proved that v satisfies the characterization of Lemma 4.5, and can conclude that $v(D) \models \gamma(v(\bar t))$ .

$\Leftarrow$ Assume v is a consistent valuation and $v(D) \models \gamma(v(\bar{t}))$ . Let s,s’ such that $D \models equiv_\gamma(\bar{t},s,s')\wedge \neg Null(s') \wedge \neg Null(s)$ . By Lemma 4.5 we have $v(s)=v(s')$ . Moreover, s,s’ are both constants, then $s=s'$ . Hence, $D \models equiv_\gamma(\bar{t},s,s')\wedge \neg Null(s') \wedge \neg Null(s)\rightarrow s=s'$ , that is, $D \models comp_{\gamma} (\bar{t})$ .

We are now ready to define a formula capturing the inclusion of supports between two conjunctions of equality atoms, which will be a crucial ingredient in our rewriting. Let $\gamma(\bar{x})$ and $\gamma'({\bar y})$ be conjunctions of equality atoms with. We define:

$$ imply_{\gamma, \gamma'} (\bar{x},\bar{y})~:=~\forall z z'~(equiv_{\gamma'}(\bar{y},z,z')\rightarrow equiv_{\gamma}(\bar{x},z,z'))$$

Using Proposition 4.7 and Lemma 4.5, we obtain:

Proof. $\Rightarrow$ Assume $D \models imply_{\gamma, \gamma'}(\bar t, \bar t') \vee \neg comp_\gamma(\bar t)$ . If $D \models \neg comp_\gamma(\bar t)$ then by Proposition 4.7, there is no consistent valuation v such that $v(D) \models \gamma(v(\bar{t}))$ and so the implication trivially holds. Now assume $D \models imply_{\gamma, \gamma'}(\bar t, \bar t')$ , that is:

$$D \models \forall z z'~(equiv_{\gamma'}(\bar{t'},z,z')\rightarrow equiv_{\gamma}(\bar{t},z,z'))$$

We want to show that for all consistent valuations v of nulls such that $v(D) \models \gamma(v(\bar t))$ , one also has $v(D) \models \gamma'(v(\bar t'))$ . Consider a valuation v of nulls such that $v(D) \models \gamma(v(\bar t))$ . Using Lemma 4.5, it is enough to show that $\forall s,s'$ such that $D \models equiv_{\gamma'}(\bar{t'},s,s')$ one has $v(s)=v(s')$ . So assume $D \models equiv_{\gamma'}(\bar{t'},s,s')$ , by our assumption it follows that $D \models equiv_{\gamma}(\bar{t},s,s'))$ , and therefore, again by Lemma 4.5, $v(s)=v(s')$ .

$\Leftarrow$ Assume for all consistent valuations v of nulls such that $v(D) \models \gamma(v(\bar t))$ , one also has $v(D) \models \gamma'(v(\bar t'))$ . By Proposition 4.7, if there is no such valuation, then $D \models \neg comp_\gamma(\bar t)$ . So assume now there is one such valuation. This entails that in particular, there exists a consistent $v^*$ satisfying $v^*(D) \models \gamma(v^*(\bar t))$ and $v^*(s) = v^*(s')$ iff $ D \models equiv_\gamma(\bar t, s,s')$ . By our assumption we have $v^*(D) \models \gamma'(v^*(\bar t'))$ . Hence, by Lemma 4.5, $\forall s,s'$ such that $D \models equiv_{\gamma'}(\bar{t'},s,s')$ one has $v^*(s)=v^*(s')$ . By the properties of $v^*$ mentioned above, this implies $D \models equiv_\gamma(\bar t,s,s')$ . We have thus shown that $D \models imply_{\gamma, \gamma'}(\bar t, \bar t')$ .

So far, we have dealt with equality subqueries and we have characterized the emptiness and inclusion of their supports (cf. Propositions 4.7 and 4.8, respectively). We can now use this machinery to characterize the support of a BCCQ. We start by expressing membership in the support of an individual CQ:

Recall that $q(\bar{w})$ is a conjunction of relational atoms, with no constants and where each one of the free variables $\bar{w}$ has at most one occurrence in q. Thus, there exists a mapping $\nu$ of $\bar w$ over $\text{adom}(D)$ such that $D \models q(\nu(\bar w))$ and $v(\nu(\bar w))=\mu(\bar w)$ . We let $\bar s = \nu(\bar w)$ . Recall that $\bar x$ and $\bar w$ do not share variables, so we can extend the mapping $\nu$ by setting $\nu(\bar x) = \bar r$ . We thus have that $\nu(\bar x \bar w) = \bar r \bar s$ and $v(\bar r \bar s) = \mu(\bar x \bar w)$ . It follows that v is a consistent valuation for which $v(D) \models \gamma(v(\bar r\bar s))$ ; then by Proposition 4.7, we also have $D\models comp_\gamma(\bar r\bar s)$ .

$\Leftarrow$ Since $D\models q(\bar s),$ we have $v(D) \models q(v(\bar s))$ . Moreover by the hypothesis $v(D) \models \gamma (v(\bar r \bar s))$ , thus $v(D) \models Q(v(\bar r))$ .

In the remainder, we consider BCCQs $Q(\bar{x})~:=~Q_1(\bar{x}) \vee \ldots \vee Q_n(\bar{x})$ in NRV disjunctive normal form (DNF) where for all $1 \leq i \leq n$ :

$$Q_i~:=Q_{i_0}(\bar{x}) \wedge \neg Q_{i_1}(\bar{x}) \wedge \ldots \wedge \neg Q_{i_m}(\bar{x})$$

and for all $1 \leq j \leq m$ :

$$Q_{i_j}~:=\exists \bar{w}_{i_j} q_{i_j}(\bar{w}_{i_j}) \wedge \gamma_{i_j}\text{ with }\gamma_{i_j} := e_{i_j}(\bar{x}\bar{w}_{i_j})$$

For convenience, we assume w.l.o.g every conjunction of literals to be of the same length m. We can also assume without loss of generality that for each i we have $\text{adom}(\gamma_{i_j}) = \text{adom}(\gamma_{i_0})$ for all j. In fact, we can always pad any $\gamma_{i_j}$ with dummy equalities $c = c$ to extend its active domain.

Given a disjunct $Q_i$ in a BCCQ in DNF, we now define poss $_{Q_i}$ , encoding the set of possible answers to $Q_i$ , and cons $_{Q_i}$ , checking the compatibility of an answer with the negative literals in $Q_i$ .

\begin{align}poss_{Q_i}(\bar{x}\bar{w}) := q_{i_0}(\bar{w}) \wedge comp_{\gamma_{i_0}}(\bar{x}\bar{w}) \wedge cons_{Q_i}(\bar{x}\bar{w}) \nonumber\end{align}

\begin{align}cons_{Q_i}& \nonumber(\bar{x}\bar{w}) := \bigwedge_{\substack{1\leq j \leq m}} \forall \bar{w}'((q_{i_j}(\bar{w}') \wedge comp_{\gamma_{i_j}}(\bar{x}\bar{w}')) \rightarrow \neg imply_{\gamma_{i_0}, \gamma_{i_j}}(\bar{x}\bar{w}, \bar{x}\bar{w}')) \nonumber\end{align}

Using these new formulae, we show that the non-emptiness of can be expressed as the existence of a possible answer.

Moreover, we can prove the following claim:

$\Leftarrow$ Assume for all consistent valuations v, $v(D) \models \gamma_{i_0}(v(\bar r \bar s))$ implies $v(D) \models \gamma'(v(\bar t))$ . By Lemma 4.5, $v^*$ being consistent $v^*(D) \models \gamma'(v^*(\bar{t}))$ .

Now fix some arbitrary $j \geq 1$ and $\bar{s'}$ with $D\models q_{i_j}(\bar{s'})\wedge comp_{\gamma_{i_j}}(\bar{r}\bar{s'})$ . By Proposition 4.8, it follows from $D \models \neg imply_{\gamma_{i_0},\gamma_{i_j}}(\bar{r}\bar{s},\bar{r}\bar{s'}) \wedge comp_{\gamma_{i_0}}(\bar r \bar s)$ that there exists a consistent valuation v’ with $v'(D) \models \gamma_{i_0}(v'(\bar{r}\bar{s}))$ but $v'(D) \not \models \gamma_{i_j}(v'(\bar{r}\bar{s'}))$ . By the above claim $v^*(D) \not \models \gamma_{i_j}(v^*(\bar{r}\bar{s'}))$ . In summary, we have:

1. (i) $D\models q_{i_0}(\bar{s})\wedge comp_{\gamma_{i_0}}(\bar{r}\bar{s})$ and $v^*(D)\models \gamma_{i_0}(v^*(\bar{r}\bar{s}))$ and so by Lemma 4.9, we have, that is, $v^*(D) \models Q_{i_0}(v^*(\bar{r}))$ .

2. (ii) For all $1 \leq j \leq m$ and assignment $\nu'$ with $\nu'(\bar{w})=\bar{s'}$ , if $D \models q_{i_j}(\bar{s'}) \wedge comp_{\gamma_{i_j}}(\bar{r}\bar{s'})$ then $v^*(D) \not \models \gamma_{i_j}(v^*(\bar{r}\bar{s'}))$ and so by Lemma 4.9, we have, that is, for all $1 \leq j \leq m$ , $v^*(D) \models \neg Q_j(v^*(\bar{r}))$ .

This means we have for all $1 \leq i \leq n$ and so.

$\Rightarrow$ Let, so v is consistent and there is some $1 \leq i \leq n$ with: (i), (ii) for all $1 \leq j \leq m$ ,. Using Lemma 4.9 (i) implies that there exists $\bar{s}$ such that $D \models q_{i_0}(\bar{s})\wedge comp_{\gamma_{i_0}}(\bar{r}\bar{s})$ and $v(D) \models \gamma_{i_0}(v(\bar{r}\bar{s}))$ . Again by Lemma 4.9, (ii) implies that for all $1 \leq j \leq m$ and $\bar{s'}$ , if $D \models q_{i_j}(\bar{s'}) \wedge comp_{\gamma_{i_j}}(\bar{r}\bar{s'})$ then $v(D) \not \models \gamma_{i_j}(v(\bar{r}\bar{s'}))$ . This entails by Proposition 4.8 that $D \models comp_{\gamma_{i_0}}(\bar{r}\bar{s}) \wedge \neg imply_{\gamma_{i_0},\gamma_{i_j}}(\bar{r}\bar{s},\bar{r}\bar{s'})$ . This shows $D \models \bigvee_{\substack{1\leq i \leq n}} \exists \bar{w}$ $poss_{Q_i}(\bar{r}\bar{w})$ .

Now that we have defined the formula expressing for a BCCQ Q non-emptiness of (Proposition 4.10), we can easily define a rewriting for the problem CertainAnswer $_\Sigma$ (Q). To do so, we rely on the fact that $\bar r \in \mathsf{cert}_\Sigma(Q,D)$ iff.

4.3 Non-rewritability in FO

The basic starting points for our investigation was the fact that $\mathsf{cert}_\Sigma(Q,D)=Q(\mathsf{chase}_\Sigma(D))$ for a CQ Q and a set $\Sigma$ of FDs, for every database D. This remained true for unions of CQs, but failed for BCCQs, forcing us to produce a Datalog rewriting to obtain certain answers. But can a first-order rewriting be obtained instead? This would make it possible to produce certain answers using the core of SQL as opposed to its recursive features which do not always perform as well in practice.

In this section, we show that the answer, in general, is negative even for CQs (and thus for BCCQs). In the next section, however we show that such rewritings can be obtained in FO for BCCQs whenever $\Sigma$ is empty.

The main result of this section is the following.

To prove inexpressibility of $\mathsf{cert}_\Sigma(Q,\cdot)$ in FO, for each $n > 0$ we create two databases $D_n$ and $D_n'$ . In both of them, E is interpreted as a disjoint union $T_1\cup T_2$ where $T_1$ and $T_2$ are balanced binary trees of depth n in which all nodes are distinct nulls. In both A and B are singleton sets. In $D_n$ , the set A contains a leaf of $T_1$ and B contains a leaf of $T_2$ . In $D_n'$ , both A and B contain leaves of $T_1$ such that their only common ancestor in the tree is the root (in other words, they are leaves of subtrees rooted at different children of the root of $T_1$ ).

Because of the constraint $\Sigma$ , for every valuation v such that the resulting database satisfies it we have that both $v(T_1)$ and $v(T_2)$ are chains. Indeed, consider any node $\bot$ with children $\bot_1,\bot_2$ in $T_i$ . If $v(\bot_1)\neq v(\bot_2)$ then the resulting tuples $(v(\bot), v(\bot_1))$ and $(v(\bot),v(\bot_2))$ violate the constraint. Thus, $v(\bot_1)=v(\bot_2)$ and applying this construction inductively we see that $v(T_i)$ is a chain. Hence, it has a single leaf, and thus $\mathsf{cert}_\Sigma(Q,D_n')$ is true, since A and B must be interpreted as that leaf. On the other hand, $\mathsf{cert}_\Sigma(Q,D_n)$ is false, since there is a valuation v that sends $T_1$ and $T_2$ into two disjoint chains, and thus, A and B are interpreted as two distinct elements.

Assume now that $\mathsf{cert}_\Sigma(Q,\cdot)$ is rewritable as an FO sentence $\phi$ . Then, for every $n > 0$ , we have $D_n'\models\phi$ and $D_n\models\neg\phi$ . We next show that such a sentence cannot exist, thereby proving non-FO-rewritability.

Recall that in a database (with one binary relation, like considered here), a radius r neighborhood of an element a is its restriction to the set of all elements reachable from a by a path of length at most r, where the path does not take into account the orientation of edges of E (e.g. if we have E(a,b) and E(c,a) then both b and c are in the radius 1 neighborhood of a). When two neighborhoods, of elements a and b, are isomorphic, it means that there is an isomorphism between them that sends a to b. In other words, centers of neighborhoods are viewed as distinguished elements when it comes to defining neighborhoods. It is known that each first-order sentence $\psi$ is Hanf-local (Fagin et al. Reference Fagin, Stockmeyer and Vardi1995): that is, there exists a number $r > 0$ such that for any two databases $D_1$ and $D_2$ , if there is a bijection f between $D_1$ and $D_2$ such that the radius r neighborhoods of a in $D_1$ and f(a) in $D_2$ are isomorphic then $D_1$ and $D_2$ agree on $\psi$ , that is either both satisfy it or both do not.

Now let r be such a number for the sentence $\phi$ we assumed exists. Consider $D_n$ and $D_n'$ and let $T_{1a},T_{1*}$ be the subtrees of the root of $T_1$ in $D_n$ such that the first contains A while the second contains neither A not B, and let $T_{2b},T_{2*}$ be defined similarly for subtrees of the root of $T_2$ with respect to B. In $D_n'$ we define $T_{1a}',T_{1b}'$ as subtrees of the root of the tree containing A,B such that the first contains the A leaf and the second contains the B leaf, while $T_{2*}',T_{2**}'$ be the subtrees of the root of the tree having neither A nor B elements. Then, it is easy to see that the following pairs of trees are isomorphic: $T_{1a}$ and $T_{1a}'$ , $T_{2b}$ and $T_{1b}'$ , $T_{1*}$ and $T_{2*}'$ , $T_{2*}$ and $T_{2**}'$ .

We now define the bijection f as the union of those isomorphisms plus mapping roots of trees $T_i$ in D into roots of $T_i$ in D’. It is an immediate observation that if $n > r+1$ (i.e. leaves are not in the radius r neighborhood of children of roots) then f satisfies the condition that neighborhoods of a and f(a) of radius r are isomorphic. This would tell us that $D_n$ and $D_n'$ agree on $\phi$ but we know they do not. This contradiction completes the proof.

As a corollary to the proof, we obtain the following result showing that non-recursive SQL is incapable of computing $\mathsf{cert}_\Sigma(Q,D)$ in the setting of Theorem 4.14.

This is due to the fact that queries in this fragment of SQL with grouping and aggregation can be translated into a logic with aggregate functions (Libkin Reference Libkin2003) which itself is known to be Hanf-local (Hella et al. Reference Hella, Libkin, Nurmonen and Wong2001).

4.4 FO rewriting for certain answers

We now focus on the special case where $\Sigma$ is empty. First notice that the only Datalog component in our rewriting was the $equiv_{\gamma}$ formula; moreover, notice that without constraints, $equiv_{\gamma}$ simply computes a reflexive symmetric transitive closure. More precisely, for a given $\bar t = \nu(\bar y)$ , one has that $equiv_{\gamma}(\bar t, s, s')$ holds in D iff (s,s’) belongs to the reflexive symmetric transitive closure of $\{(\nu(x), \nu(w)) ~|~ x = w \in \gamma(\bar y)\}$ over $\text{adom}(D) \cup\text{adom}(\gamma)$ .

As $\Sigma$ is empty, we can rewrite as follows the $equiv_{\gamma}$ formula in FO, where m is the number of equivalence classes of $\sim_{\gamma}$ :

Intuitively, this holds because each disjunct of $equivFO_\gamma(\bar t, s, s') $ corresponds to a possible derivation of (s, s’) in the reflexive symmetric transitive closure of $\{(\nu(x), \nu(w)) ~|~ x = w \in \gamma\}$ , and one can prove that there is a bound only depending on $\gamma$ on the number of steps of this derivation.

Assume now that $s \equiv_\gamma^\nu s'$ . If $s = s'$ then clearly $D \models equivFO_{\gamma} (\bar{t},s,s')$ . Thus, assume $s \neq s'$ . We proceed by induction on the number of transitive closure steps needed to derive (s,s’) starting for the base relation $\{(\nu(x), \nu(w)) | x = w \in \gamma\}$ . In the base case, $(s,s') = (\nu(x), \nu(w))$ for some equality $x=w \in \gamma$ . Then, D satisfies the following disjunct of $equivFO_{\gamma} (\bar{t},s,s')$ : take $u_1 = x$ , $v_1 = w$ , $u_i = v_i = w$ for all $i = 2..m$ (this is a disjunct since $u_i \sim_\gamma v_i$ for all $i = 1..m$ ). The disjunct is satisfied since $s = \nu(x) = \nu(u_1)$ , $s' = \nu(w) = \nu(v_m)$ , and for all $i = 1..m-1, \nu(v_i) = \nu(u_{i+1}) = \nu(w)$ .

In the general inductive case, there exists r such that $(r, s') = (\nu(x), \nu(w))$ for some equality $x = w$ (or $w=x$ ) $\in \gamma$ , with $s \equiv_\gamma^\nu r$ derived at the previous step. By the induction hypothesis $D \models equivFO_\gamma(\bar t, s, r)$ . We can assume $s \neq r$ since otherwise (s,s’) would be in the base relation. Therefore, D satisfies one of the disjuncts of $equivFO_\gamma(\bar t, s, r)$ . Then, there exists a sequence of $m+1$ pairs in $\bar y \cup \text{adom}(\gamma)$

$$ (u_1, v_1) (u_2, v_2) \dots (u_m, v_m) (u_{m+1}, v_{m+1})$$

such that

• $u_{m+1} = x$ and $v_{m+1} = w$ ,

• $u_i \sim_\gamma v_i, i =1..m+1$ ,

• $s = \nu(u_1)$ , $r = \nu(v_m)$ , $s' = \nu(v_{m+1})$ ,

• $\nu(v_i) = \nu(u_{i+1})$ , for all $i \leq m$ ,

We now show that from this sequence of pairs, one can construct another one of exactly m pairs, $(u'_i, v'_i), i =1..m$ still connecting s ans s’, that is such that:

1. (a) $u'_i \sim_\gamma v'_i, i =1..m$

2. (b) $s = \nu(u'_1)$ , $s' = \nu(v'_{m})$

3. (c) $\nu(v'_i) = \nu(u'_{i+1})$ , for all $i < m$ .

The idea is to first cut the sequence $(u_i, v_i), i =1..m+1$ , removing at least one pair, then pad it to size m if necessary.

In order to cut the original sequence, remark that it contains $m+1$ pairs where m is the number of $\sim_\gamma$ equivalence classes. Thus, there exist $i < j$ such that $u_i \sim_\gamma u_j$ . We remove from the sequence all elements between $u_i$ and $v_j$ (excluded), the new sequence is

$$(u_1, v_1) ... (u_{i-1}, v_{i-1}) (u_i, v_j) (u_{j+1}, v_{j+1}) ...(u_{m+1}, v_{m+1})$$

Note that this sequence satisfies (a) (b) and (c) above since $u_i \sim_\gamma u_j \sim_\gamma v_j$ . Let the new sequence contain k pairs. We know $k \leq m$ because we have removed at least one pair from the original sequence (recall $i < j$ ). If $k < m,$ we pad the sequence on the right with $m-k$ pairs $(v_{m+1}, v_{m+1})$ . The new sequence still satisfies (a), (b), and (c); therefore, the corresponding disjunct of $equivFO_\gamma(\bar t, s, s')$ is satisfied by D.

As a consequence of Proposition 4.17, for fixed $\gamma$ and $\bar t$ , the relation $\{(s,s') ~|~ D \models equivFO_{\gamma}(\bar t, s, s')\}$ is an equivalence relation over $\text{adom}(D) \cup \text{adom}(\gamma)$ where each element of $\text{adom}(D)$ neither in $\bar t$ nor in $\text{adom}(\gamma)$ forms a singleton equivalence class.

As in Section 4.2, $ equivFO_{\gamma}$ selects precisely the pairs of elements of a tuple that a valuation needs to collapse to satisfy a set of equalities.

As a consequence, we can rewrite in FO the formula poss $_{Q_i}$ of Subsection 4.2 encoding the set of possible answers to $Q_i$ . It is enough to replace each occurrence of the Datalog $equiv_{\gamma}(\bar{y},z,z')$ program in it by $equiv^{FO}_{\gamma}(\bar{y},z,z')$ . We denote by poss $^{FO}_{Q_i}$ the rewriting so obtained.

Note that tractability of BCCQ was already proved in Gheerbrant and Libkin (Reference Gheerbrant and Libkin2015) using tableau-based methods. We now refine complexity as follows.

4.5 FO rewriting for best answers

Considering arbitrary ${{{{\mathsf{FO}}}}}$ queries brought us an intrinsic intractability result for all variants of best answers. This motivates restricting to unions of conjunctive queries, for which a polynomial-time evaluation algorithm (in data complexity) already exists(Libkin Reference Libkin, den Bussche and Arenas2018). The resolution-based procedure is however in sharp contrast with naïve evaluation, which allows to compute certain answers to unions of conjunctive queries via usual model checking. We thus initiate a descriptive complexity analysis of the best answers problem, showing that for unions of conjunctive queries, it can essentially be reduced – modulo a preprocessing of the query – to (naïve) evaluation of an ${{{{\mathsf{FO}}}}}$ -formula.

Given a union of conjunctive queries Q, our starting point towards an ${{{{\mathsf{FO}}}}}$ rewriting for best answers is finding an ${{{{\mathsf{FO}}}}}$ -formula $Q_\subseteq(\bar{x}, \bar{y})$ encoding the inclusion of supports, that is selecting tuples $\bar s, \bar t$ over $\text{adom}(D)\cup\text{adom}(Q)$ iff. From $Q_\subseteq,$ one can easily define an ${{{{\mathsf{FO}}}}}$ -formula selecting precisely all best answers to Q on D:

(1) \begin{align}best_Q (\bar x) := \forall \bar{y}(Q_{\subseteq}(\bar{x}, \bar{y}) \rightarrow Q_{\subseteq}(\bar{y}, \bar{x}))\end{align}

As in Section 4.2, we assume all CQs to be in NRV normal form. We can thus first concentrate on the support of equality subqueries. This will be encoded in ${{{{\mathsf{FO}}}}}$ and then integrated in the rewriting of the whole conjunctive query.

We now go back to an arbitrary union of conjunctive queries of vocabulary $\sigma$ in NRV-normal form:

\begin{align}Q(\bar{x}):=\bigvee\limits_{1 \leq i \leq n}Q_i(\bar{x})\nonumber\end{align}

where each $Q_i$ is in NRV normal form with relational subquery $q_i(\bar y_i, \bar z_i)$ and equality subquery $eq_i(\bar x, \bar y_i, \bar z_i)$ .

Recall the formula $comp_{\gamma}$ , defined in Section 4.2, stating the existence of a valuation that collapses all equivalent elements of a tuple:

$$comp_{\gamma}(\bar{y})~:=~\forall z z'(equivFO_{\gamma}(\bar{y},z,z') \wedge \neg Null(z) \wedge \neg Null(z')\rightarrow z=z')$$

Notice that if $D \models comp_\gamma(\bar t)$ then for each $s \in adom(D)\cup\text{adom}(\gamma)$ there exists at most one constant c such that $D \models equivFO_\gamma(\bar t, s, c)$ . In fact if for constants $c_1$ and $c_2$ , $D \models equivFO_\gamma(\bar t, s, c_1)$ and $D \models equivFO_\gamma(\bar t, s, c_2)$ , by transitivity $D \models equivFO_\gamma(\bar t, c_1, c_2)$ , implying $c_1 = c_2$ .

We now define a formula capturing the inclusion of supports between two conjunctions of equality atoms, which will be a crucial ingredient in our rewriting.

Let $\gamma(\bar{x})$ and $\gamma'({\bar y})$ be conjunctions of equality atoms with. We define:

$$imply_{\gamma, \gamma'} (\bar{x},\bar{y})~:=~\forall z z'~(equivFO_{\gamma'}(\bar{y},z,z')\rightarrow equivFO_{\gamma}(\bar{x},z,z'))$$

By combining Propositions 4.8 and 4.7, we get:

We are now ready to define the ${{{{\mathsf{FO}}}}}$ -formula encoding the inclusion of supports.

\begin{align}Q_\subseteq(\bar{x}, \bar{x'}) := \bigwedge\limits_{1\leq i \leq n} ~~(&\forall \bar{y}\bar{z} ((q_i(\bar{y},\bar{z})\wedge comp_{eq_i}(\bar{x},\bar{y},\bar{z})) \rightarrow\nonumber\\&\bigvee\limits_{1 \leq j \leq n}\exists \bar{y}' \bar{z}'(q_j(\bar{y}',\bar{z}') \wedge imply_{eq_i, eq_j}(\bar{x}\bar{y}\bar{z}, \bar{x}'\bar{y}'\bar{z}') )~ )~)\nonumber\end{align}

Combining Lemmas 4.5, 4.9, Propositions 4.7, 4.8, and Corollary 4.23, we get:

$\Leftarrow$ Assume $Supp(Q, D, \bar s) \subseteq Supp(Q,D, \bar t)$ and let $i, \bar{a}$ , $\bar{b}$ with $D \models q_i(\bar{a},\bar{b})\wedge comp_{eq_i}(\bar{s},\bar{a},\bar{b})$ . By Proposition 4.7, there exists a valuation v (that we assume w.l.o.g. to be tight) such that $v(D) \models eq_i(v(\bar{s}\bar{a}\bar{b}))$ and so by Lemma 4.9 $v \in Supp(Q,D,\bar{s})$ . Hence by our assumption, we also have $v \in Supp(Q,D,\bar{t}),$ and so by Lemma 4.9, there exists $j,\bar{a}'\bar{b}'$ with $D \models~ q_j(\bar{a}'\bar{b}') \wedge comp_{eq_j}(\bar{t}\bar{a}'\bar{b}')$ and $v(D) \models eq_j(v(\bar{t}\bar{a}'\bar{b}'))$ .

$\Leftarrow$ Assume for all valuations v, $v(D) \models \gamma(v(\bar t))$ implies $v(D) \models \gamma'(v(\bar t'))$ . By Lemma 4.5, $v^*$ satisfying the conditions of Lemma 4.7, we have $v^*(D) \models \gamma(v^*(\bar{t}))$ and so by our assumption $v^*(D) \models \gamma'(v^*(\bar{t'}))$ .

As v satisfies the conditions of Lemma 4.7, by Claim 4.25 it follows from $v(D) \models eq_j(v(\bar{t}\bar{a}'\bar{b}'))$ that $\forall v$ with $v(D) \models eq_i(v(\bar{s}\bar{a}\bar{b}))$ , also $v(D) \models eq_i(v(\bar{t}\bar{a}'\bar{b}'))$ . Now by Proposition 4.8 $D \models imply_{eq_i, eq_j}(\bar{s}\bar{a}\bar{b}, \bar{t}\bar{a}'\bar{b}') \vee \neg comp_{eq_i}(\bar{s},\bar{a},\bar{b})$ . But $D \models comp_{eq_i}(\bar{s},\bar{a},\bar{b})$ , so $D \models \exists \bar{y}\bar{z}(~q_j(\bar{y},\bar{z}) \wedge imply_{eq_i, eq_j}(\bar{s}\bar{a}\bar{b}, \bar{t}\bar{y}\bar{z}))$ .

ecall that from $Q_\subseteq$ one can easily define a first-order rewriting $best_Q (\bar x)$ for best answers as in (1).

Example 4.27 For $Q,D,\gamma, \gamma'$ as in Examples 2.1 and 4.22:

$$Q_\subseteq(x,x') := \forall y_1y_2z ((R(y_1)\wedge S(y_2,z) \wedge comp_\gamma(y_1,y_2,z,x))$$

$$\rightarrow$$

$$ ~\exists y_1' y'_2 z'(R(y'_1) \wedge S(y_2',z') \wedge imply_{\gamma, \gamma'}(y_1y_2zx,y'_1y'_2z'x') )).$$

This allows to derive for instance (as observed in Example 2.1). In fact the subquery $R(y_1)\wedge S(y_2,z)\wedge comp_\gamma(y_1,y_2,z,x)$ with free variables $y_1,y_2,z,x$ selects on D tuples $(1,\bot_2,\bot_2,1),(\bot_1,\bot_2,\bot_2,1)$ , and no other tuple with last element 1. Moreover, as shown in Example 4.22

\begin{align}&D \models imply_{\gamma\gamma'}(\bot_1\bot_2\bot_21,1\bot_2\bot_2\bot_2)\nonumber\\&D \models imply_{\gamma\gamma'}(1\bot_2\bot_21,1\bot_2\bot_2\bot_2)\nonumber\end{align}

Thus, $D \models Q_\subseteq(1,\bot_2)\nonumber$ . Similarly, one can show $D \models Q_\subseteq(\bot_1,\bot_2),$ and therefore $D\models best_Q(\bot_2)$ .

As a corollary of Theorem 4.26, for a union of conjunctive queries Q one can compute by first computing the formula $best_Q(\bar x)$ from Q, then evaluating $best_Q$ on D. Since data complexity of ${{{{\mathsf{FO}}}}}$ query evaluation is DLogSpace (and in particular AC $^0$ ), this gives the following corollary:

Note that it was known from Libkin (Reference Libkin, den Bussche and Arenas2018) that the data complexity of computing best answers for unions of conjunctive queries is polynomial time. In terms of combined complexity (i.e. when either Q, D and $\bar a$ are in the input), the rewriting approach (i.e. the procedure of computing $best_Q$ from Q and then evaluating $best_Q$ on D), can be easily shown to be in PSPACE. In fact it is well known that a first-order query $\phi$ can be evaluated on a database D in space at most $qr(\phi) \log |D| + log|\phi|$ , where $qr(\phi)$ is the quantifier rank of $\phi$ . Note that although $best_Q$ has size exponential in Q, the quantifier rank of $best_Q$ is linear in the size of Q. Thus whether $\bar a \in best(Q,D)$ can be checked using space $O(|Q|, |D|)$ . Moreover one can easily check that $best_Q$ can be computed from Q in space polynomial in the size of $|Q|$ . Since space bounded computations can be composed without storing the intermediate output, computing $best_Q$ from Q and then evaluating $best_Q$ on D can be done overall in PSPACE in the size of $|Q|$ and $|D|$ . The rewriting approach thus implies a PSPACE upper bound for the combined complexity of BestAnswer $_\Sigma$ for unions of conjunctive queries. However, we can show that the problem actually stands in the third level of the polynomial hierarchy.

On input given by a database D, a UCQ Q, and a tuple $\bar a$ one can check as follows. First guess a tuple $\bar b$ over $\text{adom}(D)$ of the same arity as $\bar a$ ; then, using two calls to a $\Sigma^p_2$ oracle, check that $Supp(Q,D,\bar a) \subseteq Supp(Q,D,\bar b)$ and $Supp(Q,D,\bar b) \nsubseteq Supp(Q,D,\bar a)$ .

For hardness, we reduce from $\forall\exists\forall 3DNF$ , which is known to be $\Pi^p_3$ -complete (Schaefer and Umans Reference Schaefer and Umans2002). We take as input a $\forall\exists\forall 3DNF$ -formula of the form

$$F:= \forall z_1,\ldots z_l \exists x_1 \ldots x_k \forall y_1 \ldots y_p~\bigvee^n_{i=1}conj_i$$

where the each $conj_i$ is a conjunction of 3 (not necessarily distinct) literals over variables $z_1,\ldots z_l, x_1, \ldots, x_k, y_1, \ldots, y_p$ .

We construct a database $D_F$ with $\text{adom}(D_F) = \{0, 1, good, bad\}\cup \{i, \bar i, \bot_i, \bar \bot_i, | i=1..k\}$ , and a conjunctive query $Q_F(z_1, ..z_l, z)$ such that if and only if F is true.

$D_F$ is of signature $\{S^4,C^2,A^2,B^3\}$ as follows:

• The extension of S and A and B is fixed and does not depend on F:

  1. - S contains tuple (1,1,1,good), and tuples $(b_1,b_2,b_3,good)$ and $(b_1,b_2,b_3,bad)$ for every $b_1,b_2,b_3 \in \{0,1\}$ with $(b_1,b_2,b_3)\neq (1,1,1)$ . Intuitively, S encodes the possible truth assignment of each disjunct of F. Note that only the satisfying assignment (i.e. (1,1,1)) appears together with the only constant good, all the others appear both with good and bad.

  2. - A contains only two tuples: (0,1) and (1,0). Intuitively, A will be used to encode truth values for pairs of literals $(w, \neg w), w \in y_1, \ldots y_p, z_1, \ldots z_l$ .

  3. - B contains tuples (0,0, bad), (1,1, bad) and tuples $(b_1, b_2, good)$ and $(b_1, b_2, bad)$ for every $b_1, b_2 \in \{0,1\}$ , $b_1 \neq b_2$ . Intuitively, B encodes assignments for pairs of literals $(w, \neg w), w \in \{x_1, \ldots x_k\}$ . Note that here inconsistent pairs (i.e. same truth value) are possible, but these are the only ones which do not appear together with constant good.

• The extension of C depends on F and contains tuples $\{(\bot_i, i) | i=1..k\}$ and $\{(\bar \bot_i, \bar i) | i=1..k\}$ . Intuitively, a valuation (b, i) (resp. $(b, \bar i)$ ) of one of these tuples, with $b \in \{0,1\}$ , will encode truth value b for the literal $x_i$ (resp, $\neg x_i$ ) of F.

$Q_F$ is defined as follows. For each variable w of F, the conjunctive query $Q_F$ will use variables w and $\bar w$ (either quantified or free). For a literal $\alpha$ of F, the corresponding variable of $Q_F$ will be denoted as $enc(\alpha)$ . More precisely if $\alpha = w$ is a positive literal, then, $enc(\alpha):=w$ , otherwise if $\alpha = \neg w$ then $enc(\alpha) := \bar w$ .

$$\begin{array}{ll} Q_F(z_1, \ldots z_l, z) := & \exists x_1, \ldots x_k, \bar x_1, \ldots \bar x_k, y_1, \ldots y_p, \bar y_1, \ldots \bar y_p, \bar z_1, \ldots \bar z_p \\ \\& \bigwedge_{i=1,..k} B(x_i, \bar x_i, z) \wedge \bigwedge_{i=1,..p} A(y_i, \bar y_i) ~ ~\wedge \bigwedge_{i=1,..l} A(z_i, \bar z_i) ~ ~\wedge \\\\& \bigwedge_{i=1,..k} (C(x_i, i) \wedge C(\bar x_i, \bar i)) ~ ~\wedge \\\\& \bigwedge_{(\alpha_1 \wedge \alpha_2 \wedge \alpha_3)\in F} S(enc(\alpha_1), enc(\alpha_2), enc(\alpha_3), z) \end{array} $$

We can prove that all tuples of the form $(\bar t, good)$ (which we refer to as good tuples) have the same support. This is given by the set of all consistent boolean valuations (i.e. valuations of $\bot_i, \bar\bot_i$ in $\{0,1\}$ such that $v(\bot_i) \neq v(\bar \bot_i)$ for all i). Moreover we can prove that if there exists a $(\bar t, bad)$ whose support contains all consistent boolean valuations then the support of $(\bar t, bad)$ strictly contains the support of good tuples. Therefore, any good tuple (including $(\bar 0, good)$ ) is a best answer iff for all tuples $\bar t$ there exists a consistent boolean valuation which is not in the support of $(\bar t, bad)$ . We can finally show that the last holds iff F is true.

Therefore, under standard complexity theoretic assumptions, our rewriting approach is not optimal in terms of combined complexity, as it is often the case with generic approaches. However, it has the advantage of exploiting standard ${{{{\mathsf{FO}}}}}$ query evaluation, which despite the PSPACE combined complexity is highly optimized in database systems and works well in practice.