2 Definitions and prior results

Let T be a complete theory in a language L. Fix a finite relational signature V disjoint from L and let $L_V$ be the union of L by V. We write an $L_V$ structure as $(M, I)$ . Given a V structure I, the active domain of I, ${\mathsf {Adom}}(I)$ , is the union of the projections of the interpretations of symbols in V. Thus if I interprets each relation in V by a finite set of tuples, ${\mathsf {Adom}}(I)$ is a finite set. An $L_V$ formula always means an ordinary first-order formula in this signature. A first-order restricted-quantifier formula (or just “ $1$ -RQ formula”) is built up starting from first-order L formulas and V atoms as the atomic formulas. We close under Boolean connectives and quantifications of the form $\exists x \in {\mathsf {Adom}}~ \phi $ or $\forall x \in {\mathsf {Adom}} ~ \phi $ where x is a variable. The semantics on $L_V$ structure $(M,I)$ with valuation $\sigma $ is that $\exists x \in {\mathsf {Adom}}~ \phi $ holds when there is $x_0$ in ${\mathsf {Adom}}(I)$ such that $\phi $ holds on $\sigma $ extended with $x \mapsto x_0$ . The semantics of $\forall x \in {\mathsf {Adom}} ~ \phi $ is given similarly or by duality. It is easy to see that these formulas can be translated to special kinds of $L_V$ formulas: we can expand out $\exists x \in {\mathsf {Adom}} ~ \phi $ into disjunctions of formulas having the form $\exists x \mathbf {y} ~ A(x, \mathbf {y}) \wedge \phi $ for some relation $A \in V$ .

We will also focus on the case $V = \{P\}$ , abbreviating $L_V$ as $L_P$ . Here active domain quantification is just quantification over P, and we write $\exists x \in P ~ \phi $ or $\forall x \in P ~ \phi $ for such quantification. We do not need atom $P(y)$ in the base case of the syntax, since they can be mimiked by quantifications $\exists x \in P ~ x=y$ .

Example 1. Let T be the theory of an infinite dense linear order $<$ . If V contains two unary predicates P and Q, then one $L_{V}$ sentence states that there is an element above every member of Q and below every member of P:

$$\begin{align*}\exists x ~ (\forall q \in Q ~ x>q) \wedge(\forall p \in P ~ p< x). \end{align*}$$

This sentence is not $1$ -RQ. But it is equivalent to the following $1$ -RQ sentence:

$$\begin{align*}\forall p \in P ~ \forall q \in Q ~ p>q. \end{align*}$$

Note that in the prior literature many other names are used for these formulas: e.g., “active domain” or “restricted.” There, attention is often focused on theories with quantifier-elimination, and RQ formulas are built up only from atomic L formulas rather than arbitrary L formulas. In this work we will not assume quantifier-elimination in the theory, and thus use the more general definition.

An embedded finite model (over V) for theory T is a pair $(M,I)$ as above where $M \models T$ , and ${\mathsf {Adom}}(I)$ is a finite subset of the domain of M. An embedded finite subset for theory T is just the special case where $V=\{P\}$ with P unary: A pair $(M, P)$ , where $M, \models T$ and P is a finite subset of a domain. We say two formulas $\phi (\mathbf {x})$ and $\phi '(\mathbf {x})$ of $L_V$ are equivalent over T if in every embedded finite model $(M,I)$ for T, $(M, I) \models \forall \mathbf {x} ~ [\phi \leftrightarrow \phi ']$ .

We now come to our central definition.

In the prior literature $1$ -RQC is referred to just as RQC [Reference Benedikt5]: the reason for the prefix “ $1$ ” will be clear when we introduce higher-order generalizations below.

3 Higher-order collapse and high complexity evaluation

How can we obtain structures where evaluating a first-order logic formula over the model is more complicated than in $1$ -RQC structures, but not completely wild as in integer arithmetic?

We will consider a first natural weakening of RQC by loosening the notion of restricted-quantifier formula, allowing higher-order quantification over the active domain. We define higher-order sorts by starting with some base sort B and closing under tupling and power sets. The order of a sort is the maximum number of nested powersets: e.g., a sort $\mathcal {P}(\mathcal {P}(B) \times B)$ has order $2$ . The first component is a set of elements of base sort and the second is an element of base sort. Given a finite set $P_0$ , and a variable of sort $\tau $ , the valid valuations of the variables are defined in terms of the hierarchy over $P_0$ : for a variable X of sort $\mathcal {P}(\mathcal {P}(B) \times B)$ , a valid valuation of X assigns to it sets of pairs, where the first component is a set of elements of $P_0$ sort and the second is an element of $P_0$ .

Given relational schema V disjoint from L, the higher-order restricted-quantifier formulas over $L \cup V$ are built up from L formulas, atomic V formulas, and atoms $X(u_1 \ldots u_m)$ , where X and each $u_i$ are higher-order variables. The inductive definition is formed by closing under Boolean connectives and under the (implicitly) restricted quantifications $\exists X$ and $\forall X$ for X a higher-order variable. The semantics on embedded finite model $(M,I)$ is in terms of valuations over ${\mathsf {Adom}}(I)$ . For x of sort B we write out $\exists x ~ \phi $ as $\exists x \in {\mathsf {Adom}} ~ \phi $ , in order to make the bounding explicit in the syntax and to be consistent with our definition of $1$ -RQ formulas. Similarly for universal quantifiers restricted to the active domain. For X of sort $\mathcal {P}(B)$ , we write $\exists X ~ \phi $ as $\exists X \subseteq {\mathsf {Adom}} ~ \phi $ , again to make the bounding explicit. And similarly for universal quantification and at higher orders. In the case where $V=\{P\}$ , P a unary predicate, we can simplify the restricted existential second-order quantifier as $\exists X \subseteq P ~ \phi $ . For a number k, a k-restricted-quantifier formula or just k-RQ formula is one where the variables are of order at most k.

We are now ready to give our first weakening of $1$ -RQC.

Higher-order RQC has not been explored in any depth. But there is one simple observation in the literature (see [Reference Libkin35]).

It is easy to see that in considering k-RQC for any fixed k, or $\omega $ -RQC for a theory, we can restrict attention to any model of the theory. Although $\omega $ -RQC has not been studied explicitly, it is easy to see that certain examples that are not $1$ -RQC are also not $\omega $ -RQC.

A key motivation for k-RQC comes from complexity. In the context of logics over finite structures, there is a strong connection between definability in higher-order logics and complexity. For example, existential second-order definability in a language with only a relational signature—that is, no interpreted structure—corresponds to the complexity class ${\mathsf {NP}}$ . While full second-order logic, again in the setting with out interpreted structure, corresponds to ${\mathsf {PSPACE}}$ [Reference Immerman30]. In the context of finite models, higher-order definability has a correspondence with the exponential time hierarchy (see [Reference Hella and Turull-Torres22]). In the presence of interpreted structure, we can see that if $L_V$ formulas can express arbitrary k-RQC sentences, then the complexity of evaluation is (at least) non-elementary—just inheriting results from the case of finite structures. Such coarse lower bounds are independent of how the infinite structure is encoded: thus in this work, when we talk about lower bounds on evaluating sentences for a given theory, this will always refer isomorphism-invariant formulas.

We cannot claim that k-RQC implies elementary upper bounds on the complexity of evaluation for arbitrary sentences. Complexity upper bounds depend on how easy it is to evaluate quantifier-free sentences, which will depend on the specifics of the presentation of the model. But we can see the following proposition.

A partial converse will be given in Proposition 3.11.

3.1 Separating RQC levels and higher-order spectra

We will show that for any k there is a T that is $(k+2)$ -RQC but not k-RQC. We will also show that there are decidable theories that are not even $\omega $ -RQC.

As mentioned above, there is a strong connection between definability in higher-order logics and complexity. For example, existential second-order definability in a language with only a relational signature—that is, no interpreted structure, corresponds to the complexity class ${\mathsf {NP}}$ , while full second-order logic corresponds to ${\mathsf {PSPACE}}$ . Higher-order definability has a close connection to the exponential hierarchy [Reference Hella and Turull-Torres22].

In the process of separating $(k+2)$ -RQC and k-RQC, we will obtain examples of decidable structures where the complexity of evaluating formulas can be arbitrarily high.

$k^{th}$ -order logic is the logic built up using variables of order $1$ to k, with atomic formulas equality as well as $X(x)$ , where X has order $j+1$ and x order j via Boolean connectives and quantifiers $\exists S$ . We will consider “pure equality higher-order sentences”: $k^{th}$ -order logic sentences with no relational constants, only quantified variables. We can interpret such sentences over a structure consisting only of a domain—a pure set. We call a set J of numbers a pure $k^{th}$ -order spectrum if there is a sentence $\phi $ of the above form such that $J= \{n ~ \mid \{1 , \ldots n\} \models \phi \}$ .

The following is a variation of results in [Reference Hella and Turull-Torres22].

Proposition 3.5. For any k, there is a set $J_k$ of numbers that is a pure $k+2$ -order spectrum but not a pure k-order spectrum.

Note that for formulas with only equality, there is a one-to-one correspondence between models of the formula and integers, where the correspondence maps a model to its size. Thus it is enough to separate the expressiveness of $k+2$ -order logic from that of k-order logic, over a unary signature. This last follows from results in [Reference Kolodziejczyk32, Reference Mostowski, Rojszczak, Cachro and Kurczewski38], stating that a truth definition for a k-order logic sentence can always be expressed as $k+2$ -order logic sentence. This holds for any fixed signature, hence also for the signature with only equality.

Note that [Reference Hella and Turull-Torres22, Reference Hull and Jianwen28] claim separation results for expressibility in higher-order logic, following from complexity characterizations of the definable sets in the logic: in [Reference Hella and Turull-Torres22] the characterization is via non-deterministic iterated exponential time with an oracle for a class in the polynomial hierarchy. Similar characterizations are provided in [Reference Kolodziejczyk32]. In [Reference Hella and Turull-Torres22], the argument that is sketched for separation does not apply to a signature with only equality. In [Reference Hull and Jianwen28] the argument for separation relies on unpublished results in a Ph.D. thesis [Reference Bennett7], so it is difficult to say the signatures to which it applies. But it seems likely that the separation result does not apply to the case of a trivial signature.

We now show how to generate theories $T_J$ from a set of numbers J, such that $T_J$ is k-RQC if and only if J is a pure k-order spectrum. Thus separation of RQC levels corresponds to separation of higher-order definability in finite model theory. As a consequence we will have the following theorem.

Theorem 3.6. For each k there are decidable complete theories $T_k$ that are $(k+2)$ -RQC but not k-RQC.

The recipe for obtaining $T_J$ from J can be used to reduce other separation statements (e.g., about the variation of k-RQC where only Monadic higher-order quantification is allowed) to separations in finite model theory. We use a construction due to Henson [Reference Henson23], first used to show that there are uncountably many homogeneous directed graphs. See [Reference Macpherson36] for more background on these constructions.

A graph is decomposable if there is a non-trivial set S of vertices with the property that for each pair $x, y \in S$ and every other vertex v in the graph, there is an edge from x to v if and only there is an edge from y to v and similarly for edges from v to x. That is, elements of S behave the same way with respect to elements outside of S. A graph is indecomposable if it is not decomposable. What is achieved by the construction of Henson can be stated as follows.

Theorem 3.7 [Reference Henson23]

For each natural number there is a finite indecomposable directed graph $B(n)$ such that for $n \neq n'$ , $B(n)$ does not embed as an induced substructure of $B(n')$ . Furthermore, there is a first-order interpretation $\delta $ whose input language is just a binary language $<$ , such that applying the interpretation to a linear order of size n gives $B(n)$ .

A first-order interpretation of dimension d [Reference Hodges25] is a function from structures to structures, specified by a formula $\phi _{{\mathsf {Dom}}}(x_1 \ldots x_d)$ in the input vocabulary describing the domain of the output, along with, for each k-ary relation symbol in the output vocabulary, a $(k \cdot d)$ -ary formula in the input vocabulary. An interpretation allows a formula also for equality—meaning that the domain of the output is actually a quotient of the k-tuples satisfying $\phi _{{\mathsf {Dom}}}$ . But such a quotient is not needed in the interpretations of Theorem 3.7.

Let a cone be a digraph of the form $\{d\} \times D$ or $D \times \{d\}$ . One can verify that the construction of [Reference Henson23] has the property that no $B(n)$ is an induced substructure of a cone.

We consider a signature consisting of ternary relations R and Z. Given a structure M for such a relation and a in the domain of M we let $R_{a}$ be the binary relation such that $R_{a}(b,c)$ holds exactly when $R(a, b,c)$ holds. Let $Z_a$ be a binary relation defined analogously.

For a set of numbers J, let $K_J$ be the set of finite structures for $R,Z$ such that $R(x,y,z)$ implies $x,y, z$ are all distinct and $B(n)$ for $n \in J$ does not embed as an induced substructure of $R_a$ , for any a in the domain. This set of structures is easily seen to have the hereditary, amalgamation, and joint embedding properties. Thus there is a Fraïssé limit (see Section 2) which we denote as $M_J$ . Let $T_J$ be the complete theory of $M_J$ . Since there was no restriction on Z, in $M_J$ the relation Z will simply be a random ternary relation. Whenever J is a decidable set, the class of finite structures $K_J$ is effective, and thus the theory $T_J$ is computably axiomatizable via the usual “extension axioms,” stating that for every tuple $\mathbf {t}_1$ in the model and isomorphism $h_1$ of a structure $s_1$ in $K_J$ to the substructure of the model induced on $\mathbf {t}_1$ , for every structure $s_2$ such that $s_1$ is an induced substructure of $s_2$ , there is a tuple $\mathbf {t}_2$ extending $\mathbf {t}_1$ and isomorphism of $s_2$ to $\mathbf {t}_2$ . Since $T_J$ is a complete theory with a computable axiomatization, it is decidable.

The key property of $T_J$ is that for any finite linear order O over a domain $D_O$ and any a in a model M of $T_J$ , $\delta (O)$ can be embedded in $R_a$ if and only if $|D_O| \not \in J$ .

Proposition 3.8. For $k \geq 2$ , if $T_J$ is k-RQC then J is a pure k-order spectrum.

Proof. Consider the sentence $\psi (P)$ in $L_P$ stating that:

For some a and some b, $Z_{b}$ induces a linear order $<_P$ on P, and $\delta $ applied to $<_P$ is an induced substructure of $R_a$ .

The properties of the structure guarantee that such a sentence is true if and only if $|P| \not \in J$ .

Suppose $T_J$ is k-RQC. The sentence $\psi $ is cardinality-invariant. We first consider embedded finite subsets living within a totally indiscernible set I. Then there is a k-RQ sentence that defines $\psi $ , and indiscernibility allows the references to the ambient structure to be eliminated from $\psi $ resulting in $\psi '$ in pure $k^{th}$ -order logic. The sentence $\psi $ is cardinality-invariant, thus $\psi $ is equivalent to $\psi '$ on arbitrary embedded finite structures. From this we see that J is the spectrum of a $k^{th}$ -order logic sentence. In general we know only that there is a model with an order-indiscernible set: we can proceed as above (since $k \geq 2$ ) but guess the ordering with second-order quantification.

Proposition 3.9. For $k \geq 2$ if J is a pure $k^{th}$ -order logic spectrum, then $T_J$ is k-RQC.

Proof. Suppose J is the spectrum of $\tau $ .

For $G \subseteq P^2$ , we write $R_{x,1,P}=G$ to abbreviate the formula

$$\begin{align*}\forall y,z \in P ~ R(x,y,z) \leftrightarrow G(y,z). \end{align*}$$

We let $R_{x,2,P}=G$ abbreviate

$$\begin{align*}\forall y,z \in P ~ R(y,x,z) \leftrightarrow G(y,z) \end{align*}$$

and define $R_{x,3,P}=G$ , $Z_{x,i,P}=G$ for $i=1,2,3$ analogously.

Inductively, it suffices to show that a formula $\gamma (x, \mathbf {y})$ of the form:

$$\begin{align*}\exists x ~ \rho(x, \mathbf{y}) \end{align*}$$

with $\rho\ k$ -RQ, is equivalent to a k-RQ formula.

We can assume that x occurs in $\rho $ only in L atoms. Since $T_J$ enforces that $R(x,y,z)$ implies $x,y,z$ distinct, we can also assume x occurs at most once in each R atom. We then rewrite $\rho $ as

$$ \begin{align*} \exists ~ G_1, G_2, G_3 Z_1, Z_2, Z_3 \subset P^2 ~ R_{x,1,P}=G_1 \wedge \\ R_{x,2,P}=G_2 \wedge R_{x,3,P}=G_3 \wedge \cdots \wedge \rho'. \end{align*} $$

Here $\ldots $ indicates additional equalities with $Z_{x,i,P}$ . And $\rho '$ is obtained from $\rho $ by replacing each R atom containing x with the appropriate $G_i$ atom and similarly for Z. It will suffice to simplify the formula

$$ \begin{align*} \exists x ~ R_{x,1,P}=G_1 \wedge R_{x,2,P}=G_2 \wedge R_{x,3,P}=G_3 \wedge \cdots, \end{align*} $$

where $G_1, G_2$ , and $G_3$ are constrained to be in $P^2$ .

We claim that for $G_1, G_2, G_3 \subseteq P^2$ and P finite, there is x such that $R_{x,1,P}=G_1$ , $R_{x,2,P}=G_2$ , and $R_{x,3,P}=G_3$ and the $Z_{x,i,P}$ equalities hold exactly when no $B(n)$ for $n \in J$ embeds as an induced substructure of $G_1$ . This is easily verified: the fact that $G_2$ and $G_3$ play no role uses the assumption that no $B(n)$ is contained in a cone. The equalities involving Z can always be achieved.

Note that if $G_1$ contains some $B(n)$ then necessarily $n \leq |P|$ , since $G_1$ is contained in $P^2$ . Thus the condition that $G_1$ does not contain any $B(n)$ can be expressed as:

for no subset $P'$ of P, for no linear order $<$ on $P'$ if $P'$ satisfies $\tau $ , then $\delta (<)$ does not embed as an induced substructure of $G_1$ .

Since $\tau $ is a k-RQ formula in P, this is a k-RQ formula on $G_1, P$ .

Thus we have reduced separating levels of RQC to the corresponding separations in finite model theory. By Proposition 3.5 we can choose J to be the spectrum of a $k+2$ sentence but not a k-order logic sentence, and thus get theories that are $k+2$ -RQC but not k-RQC. By adjusting the definability of the set of natural numbers J, we can also show the following corollary.

Corollary 3.10. There are decidable theories that are not $\omega $ -RQC.

Proof. Choose J that is decidable, but not a pure k spectrum for any k.

By varying the complexity of the set J (while retaining decidability) we can show that the complexity of evaluating formulas can be arbitrarily high.

Recall from Section 2 that there is a strong connection between various classical model-theoretic dividing lines and $1$ -RQC: for example, $1$ -RQC implies NIP. The result above can be seen as a negative one about similar connections between higher-order collapse and classical model-theoretic dividing lines: Note that model-theoretically the different theories $T_J$ are quite similar. But they can be adjusted to get any level of RQC or none at all, depending on J.

The construction does not give theories that are NIP. We can show that it is possible to separate $2$ -RQC from $1$ -RQC with an NIP theory (see Appendix H). But the problem of separating $k+2$ from k with an NIP theory for $k>0$ is open.

4 Complexity disconnects and weakening RQC by allowing expansion of the signature

In Section 3 we dealt with examples where there was some correlation between the expressiveness of formulas and complexity of evaluation. We provided examples which not only fail to have k-RQC, but examples where logical formulas are highly expressive, and thus complex to evaluate.

On the other hand, the failure of $1$ -RQC or k-RQC alone does not mean that evaluation of formulas is complex. It might be that a structure fails to have RQC ( $1$ - or k-) not because it is too expressive, but because it is not expressive enough. Like quantifier-elimination, k-RQC is sensitive to the signature. It is easy to construct theories that are “badly behaved”—not even $\omega $ -RQC—but which become $1$ -RQC when the theory is expanded.

Before we give an example of this, we give a simple tool for verifying failure of $\omega $ -RQC.

NFR formalizes the idea that there are embedded subsets distinguished by $\phi $ that are increasingly similar locally. It is easy to see the following proposition.

We now turn to the example.

The example shows another disconnect between the notion of collapse and the complexity of the theory. In the theory T above, we do not have any kind of collapse, but this is not because evaluation of formulas is too expressive, it is just because the signature was two restricted. Thus the disconnect can be fixed by adding to the signature. We thus investigate another weakening of $1$ -RQC, allowing expansion of the interpreted signature. We wish to see if adding this ability to expand gives us a closer correlation between the collapse notion and complexity.

Thus the persistently unrestricted theories are ones in which this weakening does not help us obtain RQC. Notice that the “potentially RQ” sentence $\phi _{\subseteq }$ in Example 2 is not isomorphism-invariant. This is not a coincidence.

The argument is given in Appendix D.

The proposition tells us that weakening RQC in this manner cannot allow us to use unrestricted quantification to do any new “pure relational computation.” In particular, really badly-behaved theories like full arithmetic cannot become k-RQC via augmenting the signature.

An open question is: Is every NIP theory potentially $\omega $ -RQC? One cannot use isomorphism-invariant sentences to get a counterexample, since by Theorem 2.6 from [Reference Baldwin and Benedikt2], isomorphism-invariant formulas in NIP theories are always equivalent to $1$ -RQ ones that use an additional order, and thus in particular are equivalent to $2$ -RQ ones.

Although we do not have an NIP theory that is persistently unrestricted, the main result of this section is that there are persistently unrestricted theories that are reasonably well-behaved. We show that even when isomorphism-invariant formulas are well-behaved, and the complexity of deciding the theory is elementary (see [Reference Kozen33]), the theory may be persistently unrestricted. Thus even with this weakening, failure of the (weakened) collapse does not imply high complexity of evaluation.

We will sketch an argument that there is an infinite set A such that, for embedded finite structures with active domain of the V relations in A, every $L_V$ sentence is equivalent to a Monadic second-order logic RQ formula. In fact, we will describe such a set where the elements are totally indiscernible: the L formulas $\phi (\mathbf {a})$ satisfied by tuples $\mathbf {a}$ of distinct elements are independent of the choice of $\mathbf {a}$ . Hence for P unary, isomorphism-invariant $L_P$ sentences are equivalent to first-order sentences quantifying over P.

We first consider a different way of restricting the embedded finite structures, not looking at a set A of elements which must contain the domains.

We observe the following claim.

Now take A to be an infinite antichain in a model. We argue that A has the required property. The antichain is definable within the subalgebra that it generates by a $1$ -RQ formula: it is just the atoms of that algebra. We transform $\phi $ to $\phi '$ as follows. We add to the V signature a unary predicate P, and add a conjunct $\phi _{{\mathsf {sub}}}$ saying that the domain of P is a subalgebra, and that its atoms are exactly the active domain of V. We replace references to a V relation symbol U in $\phi $ with references to the restriction of U to elements that are atoms of the domain of P. For example, if $\phi = \exists x y ~ \neg U(x,y)$ , then $\phi '$ would be $\phi _{{\mathsf {sub}}} \wedge \exists x y ~ \neg ( x \text { and } y \text { are atoms of the domain of } P \text { such that } U(x,y))$ . It is easy to verify that $\phi '$ holds on an interpretation of relational vocabulary $V \cup \{P\}$ , where P is the subalgebra generated by the active domain of the structure, exactly when $\phi $ is true on the original interpretations. By Claim 1, $\phi '$ can be converted to a $1$ -RQ formula $\phi "$ over $\mathcal C$ . Then quantifications in $\phi "$ , where quantifications range over the subalgebra generated by the active domain, can be replaced by references to subsets of the atoms, giving a Monadic second-order restricted-quantifier formula.

Summarizing, we have shown each claim in Theorem 4.5 except the last one. We now sketch the argument that the theory is persistently unrestricted.

We start by considering one particular extension ${\mathsf {ALessBA}}^<$ , which we describe as the complete theory of a particular model. Consider the class of finite Boolean algebras, where we have an ordering on the generators and induce from this the lexicographic ordering on the corresponding algebra elements. This class of models has a Fraïsse limit [Reference Kechris, Pestov and Todorcevic31], which is the model we want. In this model we have a distinguished partition of $1$ into countably many elements ordered by a discrete linear order $<$ .

We next consider the $L_P$ formula $\phi _{\bigcup }$ stating that P has an element $x_P$ that is the union of all other elements of P. We claim that $\phi _{\bigcup }$ is NFR within ${\mathsf {ALessBA}}^<$ . Let us recall the definition of an $L_V$ formula being NFR(see Definition 4.1). This means that there is a family of pairs of embedded finite subsets $P_n$ and $P^{\prime }_n$ , along with a mapping $f_n$ from $P_n$ to $P^{\prime }_n$ such that $f_n$ preserves all L formulas with at most n free variables, but each $(M,P_n)$ disagrees with $(M,P^{\prime }_n)$ on $\phi $ . Here “all L formulas” can be replaced by “all atomic L formulas,” using quantifier elimination in ${\mathsf {ALessBA}}$ . It is easy to see that ${\mathsf {ALessBA}}^<$ admits such a family. We let $P_n$ be a set of n disjoint Boolean algebra elements in the linear ordered set, along with an additional element representing their union. While $P^{\prime }_n$ consists of all but the $<$ -last of the n elements, along with the union of all the n elements. We note that NFR witnesses the failure of $\omega $ -RQC, as mentioned in Proposition 4.2.

We next make use of a result from [Reference Hrushovski27]. For a tuple $\mathbf {x}_0$ in an $L'$ structure $M'$ and $L'$ a subset of L, the L-type of $\mathbf {x}_0$ is the set of formulas $\phi (\mathbf {x})$ with vocabulary in L that are satisfied by $\mathbf {x}_0$ . A theory T is everywhere Ramsey theory if for every model M of T, for every expansion $M_e$ of M, there is an elementary extension $M^*_e$ of $M_e$ and a copy $M'$ of M such that for every k, the $L(M)$ -type of a k-tuple in $M'$ determines its type in $M^*_e$ . The result we use is Example 5.7, part 7, from [Reference Hrushovski27].

Further discussion on Ramsey properties of ${\mathsf {ALessBA}}^<$ , and the corresponding class of finite lex-ordered subalgebras, can be found in Section D, starting page 34 in [Reference Kechris, Pestov and Todorcevic31].

Now consider an arbitrary expansion $M^*$ of a model of ${\mathsf {ALessBA}}$ . We can further expand $M^*$ to a model of ${\mathsf {ALessBA}}^<$ by choosing the ordering appropriately. Let $M, P_n, P^{\prime }_n$ , and $f_n$ be the witness finite models for ${\mathsf {ALessBA}}^<$ above. Since ${\mathsf {ALessBA}}^<$ is everywhere Ramsey, we find a copy of M inside an elementary extension of $M^*$ . The copies of $P_n, P^{\prime }_n$ , and $f_n$ will witness that $M^*$ is also not $\omega $ -RQC.

5 Failure of weaker forms of collapse for finite fields and connections to complexity of decision procedures

Section 3.1 provided us with examples of theories that have different levels of higher-order collapse, and also varying complexity of evaluation. It is natural to ask about decidable theories arising naturally, both in terms of collapse and complexity. In Section 4 we saw examples of decidable structures that behave badly with respect to collapse—not even $\omega $ RQC—but this did not reflect something fundamental about complexity. This was true in the case of Example 4, since collapse could be fixed by expansion.

What about other well-studied decidable theories. In previous work it was noted that many decidable examples (e.g., real-closed fields) satisfy broad model-theoretic properties like o-minimality, known to imply $1$ -RQC. We have seen that random graph is $2$ -RQC. Another well-known source of decidable theory is the theory of finite fields: the set of sentences that hold in all finite structures. This was shown decidable by Ax [Reference Ax1]. This is an incomplete theory, but there are decidable completions, the theories of pseudo-finite fields: infinite fields that satisfy all the sentences holding in every finite field. There are several alternative definitions. For example, they are the ultraproducts of finite fields [Reference Ax1, Reference Fried and Jarden20]: a pseudo-finite field of characteristic p is an ultraproduct of finite fields of characteristic p, while a pseudo-field of characteristic $0$ is an ultraproduct of finite fields with unbounded characteristic. Pseudo-finiteness can be axiomatized, giving another decidable incomplete theory [Reference Ax1]: the assertions we make below will apply to all pseudo-finite fields—equivalently, all completions of the axiomatization.

Some of our results will focus on positive characteristic. We let $F_p$ be the theory of finite fields in characteristic p, and $PF_p$ be the theory of pseudo-finite fields in characteristic p. We begin by showing that these are not k-RQC for any k, even for isomorphism-invariant formulas. This result shows that natural decidable theories can fail to have even weak forms of RQC. It is arguably surprising given that pseudo-finite fields have some commonalities with the $2$ -RQC random graph. They are both canonical examples of “simple but unstable” (hence not NIP) theories [Reference Wagner43].

In fact, we will show that in pseudo-finite fields, and in sufficiently large finite fields, we can interpret the $n^{th}$ iterated powerset over P, for all n, uniformly in the field. From there it is easy to show, since we can code k-order logic for any k, that such fields cannot be k-RQC for any k. We will use this result to derive new lower bounds on decision procedures for pseudo-finite fields and for finite fields.

By a $k^{th}$ -order finite set theory structure over a predicate P, we mean a structure with language $(P_1,\ldots ,P_k,E_1,\ldots ,E_k)$ , with $P_1=P$ , $P_i$ a unary predicate, $E_i \subset P_i \times P_{i+1}$ a binary predicate for a membership relation, such that $E_i$ satisfies extensionality and $P_{i+1}$ can be viewed as a set of subsets of $P_i$ closed under flipping membership of one element, i.e., such that

$$ \begin{align*} \forall x \in P_i ~ \forall y \in P_{i+1} ~ \exists y' \in P_{i+1}\\ (x E_i y' \leftrightarrow \neg x E_i y) \wedge [\forall z \in P_i ~ z \neq x \rightarrow (z E_i y' \leftrightarrow x E_i y) ]. \end{align*} $$

In case the interpretation of P is finite, this implies that $(P_i,P_{i+1},E_i)$ is isomorphic to the membership relation on $P_i$ and the set of subsets of $P_i$ . Interpreting such a structure is equivalent to interpreting the $k^{th}$ iterated powerset of P.

Theorem 5.1. For any k, there exist $L_P$ formulas

$$\begin{align*}\phi_{P_1},\ldots,\phi_{P_k}, \phi_{E_1},\ldots,\phi_{E_k} \end{align*}$$

with the following properties:

• • each $\phi _{P_i}$ has one distinguished free variable, and each $\phi _{E_i}$ has two distinguished free variables. Each formula may optionally have additional “parameter variables”;

• • for any $M \models PF_p$ , and any interpretation of P as a finite subset of M, for some elements $(c_1,\ldots ,c_j)$ of M for the parameter variables the $\phi _{P_i},\phi _{E_i}$ define the full $k^{th}$ -order set theory structure over P.

Further, there is an $L_P$ formula $\phi _{\textbf {good}}$ such that the parameters can be taken to satisfy $\phi _{\textbf {good}}$ , and every parameters satisfying $\phi _{\textbf {good}}$ have the properties above.

Proof. Consider the additive group A as an ${\mathbb {F}}_p$ -vector space. We make use of the fact that there is a definable non-degenerate bilinear form ${b}$ into ${\mathbb {F}}_p$ . For example, one can use the trace product (see Lemma 3.9 of [Reference Macpherson and Steinhorn37]). Now consider $X_0$ a finite subset of A, of size $n_0$ . Define $X_i,Y_i \subset A$ inductively. We will use as parameters elements $t_1,t_2,\ldots $ , taken to be algebraically independent over the field generated by $X_0$ . We will always have $Y_i$ be the subspace generated by $X_i$ :

$$\begin{align*}y \in Y_i \iff (\forall x)[(\forall u \in X_i ~ {b}(x,u) = 0) \to {b}(x,y)=0]. \end{align*}$$

Inductively we set

$$\begin{align*}X_{i+1} = \left\{ \frac{1}{t_i-y}: y \in Y_i \right\}. \end{align*}$$

Then $X_{i+1}$ is linearly independent, and we can generate $Y_{i+1}$ as above. Note that for a non-degenerate bilinear form, the linear span of a finite set P is just the orthogonal complement of the orthogonal complement: the set of elements x such that ${b}(x,a)=0$ whenever ${b}(a,p)=0$ for each $p \in P$ . Thus there is an $L_P$ open formula that defines the linear span of a finite set P. Given a finite subset J of $X_{i+1}$ with its sum $e_J$ , we can recover J from $e_J$ as the set of $j \in J$ such that $e_J$ is not in the span of $J-\{j\}$ : this definition is correct because J was linearly independent. Composing with the inverse of the map $ \frac {1}{t_i-y}$ , we can see that $X_{i+1}$ codes subsets of $X_i$ . Thus $i^{th}$ -order monadic logic over $X_0$ is interpretable in $X_i$ , and by considering $k'$ sufficiently higher than k we can interpret full $k^{th}$ -order logic.

We now discuss how to capture the parameters by a formula $\phi _{\textbf {good}}$ as required in the last part of the theorem. Let $T_i(P)$ be the theory of $i^{th}$ -order set theory over a predicate P interpreted by a finite set. $T_i(P)$ is finitely axiomatizable. For example, for $i=1$ we have a predicate $\epsilon (x,y)$ and we need to say that there is y with no elements, and for each y and each $p \in P$ there is a $y'$ such that $y'$ agrees with y on $P \setminus \{p\}$ and disagrees with y on p. Let $good_i(\mathbf {t})$ be an $L_P$ formula expressing that the formulas $\phi _1(\mathbf {t}) \ldots $ with parameters $\mathbf {t}$ defined above (for i) give an interpretation satisfying $T_i(P)$ . Since $T_i(P)$ is finitely axiomatizable, this is a first-order sentence in $L_P$ . The result above shows that the theory of pseudo-finite fields entails $\exists \mathbf {t} ~ good_i(\mathbf {t})$ , for each i.

Note that above we talked about an interpretation with equality interpreted in the standard way, but where the formulas in the interpretation can include parameters. The standard notion of interpretation [Reference Hodges25] allows the output model to be defined as a quotient by a definable equivalence relation. That is, the interpretation can interpret equality. We can remove the notion of “good parameter” by introducing such an equivalence relation, thus talking about an interpretation without parameters. In either case, the consequence is the following corollary.

5.2 Lower bounds for quantifier-elimination

We can also make a conclusion about quantifier elimination. We will state the results for pseudo-finite fields, but they could be restated in terms of finite fields.

A parameterized algebraically-constrained formula is a formula

$$\begin{align*}\exists y_1 \ldots y_k ~ \bigwedge_i F_i(\mathbf{x}, y_1 \ldots y_k,\mathbf{p})=0, \end{align*}$$

where $F_i$ are sums of monomial terms in $\mathbf {p}, \mathbf {y}, \mathbf {x}$ , such that, in any finite field, for each $\mathbf {x}$ and $\mathbf {y}$ , the number of witnesses $\mathbf {y}$ that satisfy the formula is at most polynomial in k and the maximum degree of the $F_i$ .

Theorem 5.6. For every formula $\phi (\mathbf {x})$ in the language of fields there is a disjunction of parameterized algebraically-constrained formulas $\phi '(\mathbf {x}, \mathbf {p})$ and a formula $\theta (\mathbf {p})$ such that in every pseudo-finite field M there is some $\mathbf {p}_0$ satisfying $\theta $ , and for any such $\mathbf {p} \phi (\mathbf {x})$ is equivalent to $\phi '(\mathbf {x}, \mathbf {p})$ in M.

We provide a proof in Appendix F, but we stress that similar quantifier elimination results are known from the literature (see, e.g., Section 2 of [Reference Chatzidakis, Van den Dries and Macintyre11] and [Reference Ax1]).

Theorem 5.7. The complexity of quantifier-elimination turning a $PF_p$ formula into a positive Boolean combination of basic formulas cannot be bounded by a stack of exponentials of bounded height; this is already true for $PF_p$ for any fixed prime p. In fact for any k there exist formulas $\phi _n$ with a bounded number of variables—and hence with a bounded quantifier rank, independent of n—of length $O(n)$ , such that any basic formula $\psi _n$ which is $PF_p$ equivalent to $\phi _n$ , must have size at least $p^{p^{\cdot ^{\cdot ^{\cdot ^{p^n}}}}}$ (exponential tower of height $O(k)$ ).

Before giving the proof of the proposition, we present the main tool that we use.

We will argue the following theorem.

Theorem 5.8. There is an elementary function F such that in any pseudo-finite field if $\theta (\mathbf {x}, \mathbf {p})$ is an algebraically-constrained formula and for a given $\mathbf {p}$ the set $\theta _{\mathbf {p}}$ of $\mathbf {x}$ such that $\theta (\mathbf {x}, \mathbf {p})$ holds is finite, then $\theta _{\mathbf {p}}$ has size at most $F(size(\theta ))$ . Here size of $\theta $ takes into account the degrees of the polynomials in addition to the number of terms.

Theorem 5.8 can be thought of as an analog of Bezout’s theorem bounding the number of joint solutions of polynomial equalities. The subtlety is that while Bezout’s theorem works over algebraically closed fields, here we are working in pseudo-finite fields, which will never be algebraically closed. The above result follows from [Reference Hrushovski26, Proposition 2.2.1]. We now give the details.

Recall that an algebraically-constrained formula is an existential quantification of a conjunction of polynomial equalities, The result can be restated in terms of finite fields: if we have a basic formula $\theta $ where the size of solutions is bounded as we range over all finite fields and parameters $\mathbf {p}$ , then the size must be bounded by an elementary function. We first argue for a similar statement that removes the existential quantification.

Theorem 5.9. There is an elementary function F such that in any pseudo-finite field if $\theta (\mathbf {x}, \mathbf {p})$ is a conjunction of polynomial equalities and for a given $\mathbf {p}$ the set $\theta _{\mathbf {p}}$ of $\mathbf {x}$ such that $\theta (\mathbf {x}, \mathbf {p})$ holds is finite, then $\theta _{\mathbf {p}}$ has size $F(size(\theta ))$ . As in Theorem 5.8, size takes account the degrees of the polynomials in addition to the number of terms.

We need some auxiliary definitions. The theory of algebraically closed fields with an automorphism (ACFA) is in the language of an algebraically closed field F with a distinguished automorphism $\sigma $ satisfying certain axioms, one of which is that it is a difference-closed difference field.

The axioms will not concern us. The only thing that is important to us is as follows.

Proposition 5.10. Every pseudo-finite field is the fixed field of $(K, \sigma )$ that is a model of ACFA.

See, for example, [Reference Garcia, Macpherson and Steinhorn21], comments after Proposition 4.3.6. The proposition above allows us to reduce reasoning about polynomial equalities within a pseudo-finite field, which is not algebraically closed, to reasoning within the fixed field of an algebraically closed field with a suitable automorphism. We can thus think of our solution set as the conjunction of the polynomial equalities along with the “difference equation” $\sigma (x)=x$ .

An affine variety over a field K is a set of elements K defined by polynomial equations with coefficients in the field. So we are interested in the intersection of an affine variety with a simple difference equation.

We now state Proposition 2.2.1 of [Reference Hrushovski26].

Proposition 5.11. Let $(K; \sigma )$ be a difference-closed difference field; and let S be a subvariety of $\mathcal {P}^l$ defined over K. Let Z be the Zariski closure of

$$\begin{align*}\{x \in \mathcal{P}:(x, \sigma(x) \dots \sigma^{l-1}(x)) \in S\}. \end{align*}$$

Then $deg(Z) \leq deg(S)^{dim(S)}$ . In particular, Z has at most $deg(S)^{dim(S)}$ irreducible components.

We explain the route from Proposition 5.11 to Theorem 5.9, first introducing what we need to know about the relevant terminology from Proposition 5.11. Proposition 5.11 is stated for a subvariety of a projective variety: this is $\mathcal {P}^l$ . Again, we will not need the definition of projective varieties, but note that an affine variety can be embedded in a projective variety in a way that preserves number of solutions (by adding additional variables and equations). Thus the same proposition also holds for an affine variety S sitting within another affine variety.

Proposition 5.11 also refers to the Zariski closure, degree, and dimension. But if a variety is finite, its Zariski closure is just itself, the degree is the number of points, and the dimension is upper bounded by the number of variables in the defining polynomials.

With this in mind, we begin the proof of Theorem 5.9.

Proof. We apply Proposition 5.10, to fix an algebraically closed field K and an automorphism $\sigma $ with $(K, \sigma ) \models ACFA$ , such that our pseudo-finite field is the fixed field of K under $\sigma $ . Thus the set we are interested in is

$$\begin{align*}\{x | \{(x, \sigma(x) \dots \sigma^{l-1}(x)) \in S\}, \end{align*}$$

where S is the subvariety of the product of our variety with itself, adding the additional equation saying that the components are all equal. Proposition 5.11 tells us that the degree of this subvariety is bounded by $deg(S)^{dim(S)}$ . By the comments above about degree and dimension for finite set, this is an exponential in the size, so we are done.

We are now ready to begin the proof of Theorem 5.7.

Proof. Fix k. As in Theorem 5.1, we can write a formula $\phi ^k(x;P)$ of $L_P$ of size $O(k)$ such that whenever P is interpreted by a set of size n elements, then $\phi ^{k}(x;A)$ defines a set whose size is on the order of $n^{n^{\cdot ^{\cdot ^{\cdot ^n}}}}$ , where the ellipses denote an exponential tower of height $k'$ . For q a power of p, let $\phi _{k,q}(x) = \phi _{k}(x; x^{q}=x)$ be the formula obtained from $\phi _k$ by replacing P by $x^q=x$ . Note that $x^q=x$ has precisely q solutions, and so $\phi _{k,q}$ has on the order of $q^{q^{\cdot ^{\cdot ^{\cdot ^{q}}}}}$ solutions, where again the ellipses denote an exponential tower of height $k'$ . Note that these formulas have bounded quantifier rank.

Consider a disjunction of algebraically-constrained formulas $\psi _q$ produced by $\phi _{k,q}$ . It suffices to show: the number of solutions of such a formula is at most elementary in its size. For this, it suffices to show this for a single algebraically-constrained formula.

The projection $(x,y) \mapsto x$ maps the solutions of the atomic formulas $\theta (x, y)$ in an algebraically-constrained formula equivalent to $\psi $ onto those of $\psi $ . By the definition of algebraically-constrained, this map is at most $\ell $ to $1$ , where $\ell $ is the maximum degree of polynomials in the representation. So it suffices to bound the number of x in the quantifier-free formulas of the form $\theta $ . We obtain this via Theorem 5.9.

5.3 $\omega $ -RQC and pseudo-finite fields

We continue with the theory $T_p$ of pseudo-finite fields of positive characteristic $p>0$ . Our next goal is the following theorem.

Theorem 5.12. The theory of a pseudo-finite field of positive characteristic p is not $\omega $ -RQC.

As in our earlier examples of badly-behaved theories, we contradict $\omega $ -RQC by displaying a formula that is NFR. That is, we show that there is an $L_P$ -sentence $\psi $ such that for arbitrary large n, there are embedded finite subsets $(M,P_n)$ , $(M,P^{\prime }_n)$ with M a pseudo-finite field and a function $h_n$ from $P_n$ to $P^{\prime }_n$ that preserves all atomic formulas among n elements. Rephrased, we show that we can find isomorphic pseudo-finite substructures of some pseudo-finite field of characteristic p such that $\psi $ holds in one but not the other.

We will assume $p>2$ ; but an analogous construction works if $p=2$ , replacing the use of square roots in the argument below by Artin–Schreier roots (solutions y of $y^2+y=x$ ).

Lemma 5.13. Let k be a field of char. $p\neq 2$ . Let $K=k(x_1,\ldots ,x_n)$ be the rational function field over k in n variables. We can consider each $x_i$ as a member of the field, and let $x=\sum _{i=1}^n x_i$ . Let $K^{alg}$ be an algebraic closure of K. Within $K^{alg}$ , let $K_i$ be the algebraic closure of $k_i:=k(x_1,\ldots ,x_{i-1},x_{i+1},\ldots ,x_n)$ . Then x is not a square in $K_1 K_2 \ldots K_n$ .

Proof. For any irreducible $f \in k[X_1,\ldots ,X_n]$ , we have a valuation $v_f: K \to {\mathbb {Z}}$ , defined by

$$\begin{align*}v_f(g) = j \iff g= f^j \cdot h, \end{align*}$$

where h is a polynomial relatively prime to f. The existence and uniqueness of j follows from unique factorization in $k[X_1,\ldots ,X_n]$ , and the fact that f is irreducible. It is clear that $v_f(uv)=v_f(u)+v_f(v)$ , $v_f$ is $0$ on k, $v_f(u+v) \geq \min (v_f(u),v_f(v))$ .

Extend $v_{x_1+\cdots +x_n} $ to a valuation on $K^{alg}$ , denoted by v. The key property is that v is $0$ on K and $1$ on x.

An automorphism $\sigma $ of a Galois extension L of K over K is said to fix the residue field if whenever $v(u)=0$ , we have $v(u-\sigma (u))>0$ , i.e., $\sigma $ induces the identity on the residue field. Let $Aut(L/K,{\mathrm {res}}(L))$ denote the group of automorphisms of $L/K$ fixing the residue field. The set of elements fixed by every member of $Aut(L/K,{\mathrm {res}}(L))$ is called the maximal unramified subextension of L/K. If it equals L, we say that $L/K$ is unramified or L is unramified over K, and otherwise we say that L is ramified over K.

Now v restricts to the trivial valuation of each field $k_i$ : every non-zero element maps to $0$ . Using basic valuation theory (applying the definition of a valuation to minimal polynomials), v is trivial on each $K_i$ . Hence the only automorphism of $K_i$ over $k_i$ that fixes the residue field is the identity, and thus the same statement holds for automorphisms of $K_1\ldots K_n$ fixing the residue field. So $K_1 \ldots K_n$ is unramified over K. To prove that $\sqrt {x} \notin K_1 \ldots K_n$ , it suffices to show that $K(\sqrt {x})$ is ramified over K, which we argue next.

Since x is not a square in K (again using unique factorization), $K(\sqrt {x})$ is a Galois extension of K of degree $2$ and admits the automorphism $\tau $ fixing K and with $\tau (\sqrt {x})=- \sqrt {x}$ . Since $p \neq 2$ , $\tau $ is not the identity. We will show that $\tau $ fixes the residue field, thus $\sqrt {x}$ witnesses that $K(\sqrt {x})$ is ramified over K.

An element c of $K(\sqrt {x})$ has the form $c=a+b \sqrt {x}$ with $a,b \in K$ . If $b=0$ then $v(c-\tau (c))=v(0)=\infty $ and in particular $v(c-\tau (c)) \neq 0$ . Thus for the purposes of showing that $\tau $ fixes the residue field, we assume $b \neq 0$ . We have $v(a) \in {\mathbb {Z}}$ , and $v( b \sqrt {x}) = 1/2 + v(b) \notin {\mathbb {Z}}$ . So $v(a) \neq v(b \sqrt {x})$ . Thus $v(c) = \min \{v(a), v(b)+1/2\}$ . Towards arguing that $\tau $ fixes the residue field, we are interested in c with $v(c) =0$ . $v(b)+1/2$ cannot be $0$ , so if $v(c)=0$ , we know that $v(a)=0$ and $v(b) \geq -1/2$ . Since $v(b)$ is an integer, we conclude that $v(b) \geq 0$ . We have $v(c-\tau (c))=v(a+b\sqrt {x}- (a-b \sqrt {x}))=v(2b \sqrt {x})\geq v(b)+1/2$ . Since $v(b) \geq 0$ , we deduce that $v(c-\tau (c))>0$ as required.

This shows that $\tau $ fixes the residue field, and $K(\sqrt {x})$ is ramified over K, finishing the proof of the lemma.

Lemma 5.14. Let F be a pseudo-finite field of char. >2, and $a_1,\ldots ,a_n \in F$ algebraically independent elements. Then there exists a pseudo-finite field $F'$ and $a^{\prime }_1,\ldots ,a^{\prime }_n \in F'$ such that

$$\begin{align*}(F,a_1,\ldots,a_{i-1},a_{i+1},\ldots,a_n) \equiv (F',a^{\prime}_1,\ldots,a^{\prime}_{i-1},a^{\prime}_{i+1},\ldots,a^{\prime}_n) \end{align*}$$

for each i, while also $F \models (\exists y)(y^2 =\sum a_i)$ iff $F' \models \neg (\exists y)(y^2 =\sum a_i')$ .

Proof. Since F is pseudo-finite, it has one extension of each fixed degree, and thus the Galois group for any fixed degree is cyclic (see, e.g., [Reference Chatazadakis10]). We can thus create an automorphism of each fixed degree extension where the set of elements fixed is exactly F, and since the full algebraic closure is the union of these extensions, $F=Fix(\sigma )$ for some automorphism $\sigma $ of the algebraic closure $F^{alg}$ .

Let k be the algebraic closure of the prime field, $K=k(a_1,\ldots ,a_n)$ , $k_i = k(a_1,\ldots ,a_{i-1},a_{i+1},\ldots ,a_n)$ , and $K_i = k_i^{alg}$ . By Lemma 5.13, there exists an automorphism $\tau $ of $F^{alg}$ fixing $K_1,\ldots ,K_n$ but with $\tau (a) = -a$ , where $a=\sum _{i=1}^n a_i$ . Let $\sigma ' = \tau \sigma $ . Let K be $Fix(\sigma ')$ . Thus K is a subfield of $F^{alg}$ . Further K has exactly one extension of every degree. We now use the following result.

For F a pseudo-finite field, K a subfield of $F^{alg}$ having at most one extension of each degree, there is a pseudo-finite field $K'$ such that the algebraic numbers in $K'$ are isomorphic to K. By taking an isomorphic copy, we get that K is a subfield of $K'$ and K is relatively algebraically closed in $K'$ .

The claim above is an extension of Proposition 7 in [Reference Ax1], which states that for every subfield K of the algebraic numbers such that K has at most one extension of each degree, there is a pseudo-finite field $K'$ such that the algebraic numbers in $K'$ are isomorphic to K.

Thus there exists a pseudo-finite field $F'$ containing $Fix(\sigma ')$ and such that $Fix(\sigma ')$ is relatively algebraically closed in $F'$ . For each i set $a_i' = a_i$ . Then for any x we have $F' \models (\exists y)(y^2 =x)$ iff $Fix(\sigma ') \models (\exists y)(y^2 =x)$ . In the case $x=a$ , we have this is true iff $\tau \sigma (\sqrt {a} ) = \sqrt {a}$ iff $- \sigma (\sqrt {a}) = \sqrt {a}$ iff $\sqrt {a} \notin F$ iff $F \models \neg (\exists y)(y^2 =a)$ . To see the right to left direction, note that if $F \models \neg (\exists y)(y^2 =a)$ , then $\sigma $ cannot fix $\sqrt {a}$ , and hence it must map $\sqrt {a}$ to $-\sqrt {a}$ , the only other root of a in the algebraic closure. $\tau $ would need to send $-\sqrt {a}$ to $\sqrt {a}$ , since otherwise it must fix $\sqrt {a}$ , which means it would fix a as well, contrary to the assumption on $\tau $ . The left-to-right direction is argued similarly.

Again by results of Ax, the theory of $(F, a_1,\ldots ,a_{n-1})$ is determined by

$$\begin{align*}k(a_1,\ldots,a_{n-1})^{alg} \cap F. \end{align*}$$

This is identical to the same expression for $F'$ , so the theories are equal; and likewise for any other $n-1$ -tuple. This completes the proof of Lemma 5.14.

Note that above we have dealt with embedded finite subsets over distinct pseudo-finite fields. We can get the same result for a class over the same field: $F,F'$ are elementarily equivalent; so one can find $F"$ and elementary embeddings of F and $F'$ into $F"$ ; of course the images of the $a_i$ will be two different n-tuples in $F"$ , with the same properties as above.

We now prove Theorem 5.12. Now recall that, due to the presence of a nondegenerate bilinear form, we have $L_P$ formulas that assert:

• • P is linearly independent;

• • y is the sum of the elements of P: in particular, y depends on P and for $u \in P y-u$ depends on $P \smallsetminus \{u\}$ .

The bilinear form is used to state that y is the sum of elements of P. Let $\phi (y)$ be this formula. We also have an $L_P$ sentence $\psi $ asserting that the sum of elements of P is a square. By Lemma 5.14, for any sublanguage $L'$ of L generated by formulas of arity $n-1$ or less, there exist two $L'$ -isomorphic choices of P, one satisfying $\psi $ and the other not. From this we can easily conclude that $\psi $ cannot be equivalent to a k-RQ formula for any k. Therefore the theory is not $\omega $ -RQC.

Notice that our example contradicting $\omega $ -RQC is not isomorphism-invariant. We believe this is not a coincidence. By Proposition 3.11 if there were an isomorphism-invariant $L_V$ sentence that is not equivalent to a k-RQ one for any k, then there could be no elementary time algorithm for deciding sentences with a fixed number of variables, for any completion of theory of pseudo-finite fields. We conjecture that there are completions that are “fixed-variable elementary,” which would disprove the existence of such a sentence.

6 Embedded models vs embedded subsets

We consider one more weakening of ( $1$ -)RQC, where we restrict based on the signature for uninterpreted relations, focusing on sentences dealing with embedded finite subsets rather than general embedded finite models.

The motivation is to say whether this provides us with new collapse phenomena. We will see that it does not.

We begin by extending Proposition 2.4.

Proof. If T is not NIP, then by [Reference Simon42], we can find an L formula $R(x,y, \mathbf {p})$ such that in every model there is $\mathbf {c}$ such that there are arbitrarily large sets shattered by $R_{y,\mathbf {c}}=\{x ~ | ~ R(x; y, \mathbf {c})\}$ . For short “R has IP.” Thus, we can find an infinite order indiscernible sequence such that every subset of the sequence is $\{x | R(x, b, \mathbf {c})\}$ for some b.

We find a sequence $\langle a_i, b_i\rangle _i \in {\mathbb {N}}$ with each $a_i \neq b_i$ , $\langle a_i, b_i\rangle $ indiscernible over $\mathbf {c}$ , and for each $i, j \in {\mathbb {N}}$ , $R(a_i, b_j, \mathbf {c})$ if and only if $i<j$ .

We can do this by starting with $\langle a_i\rangle $ indiscernible over $\mathbf {c}$ , then build $b_j$ inductively: the induction step uses that every subset of the sequence is $\{x| R(x, b, \mathbf {c})\}$ for some b. Then we take a subsequence of $\langle a_i, b_i\rangle $ that is indiscernible over $\mathbf {c}$ .

We now consider several cases.

Case 1: There exists some $b^*$ such that $R(a_i,b^*, \mathbf {c})$ for all i, and for infinitely many $i \neg R(b_i, b^*, \mathbf {c})$ . Then, by refining, we can arrange that $\neg R(b_i,b^*, \mathbf {c})$ for all i and $\langle a_i, b_i\rangle $ is indiscernible over $\mathbf {c} \cup b^*$ .

We can arrange that $\mathbf {c} \cup b^*$ has cardinality $0$ mod $4$ via padding. Now we consider the class $\mathcal C$ of embedded finite subsets where P is interpreted by $P_i=A_i \cup B_i \cup \mathbf {c} \cup b^*$ .

We define the $L_P$ sentence $\phi $ to hold if there exists $\mathbf {c}, b^*$ consisting of distinct values, such that, letting:

$$\begin{align*} & A_{\mathbf{c},b*} = \{x \in P ~ | \bigwedge_i x \neq c_i \wedge x \neq b^* \wedge R(x, b^*, \mathbf{c})\} &\\ & B_{\mathbf{c}, b^*} = P \setminus A_{\mathbf{c}, b^*} \setminus b^* \setminus \mathbf{c} &\\ & <^A_{\mathbf{c}, b^*} = \{ \langle x,y \rangle ~ | x \neq y \wedge x,y \in A_{\mathbf{c}, b^*} \wedge \forall z \in B_{\mathbf{c}, b^*} (R(x,z, \mathbf{c}) \rightarrow R(y,z, \mathbf{c})\} &\\ & <^B_{\mathbf{c}, b^*} \text{ defined similarly as above but with } B_{\mathbf{c}, b^*} \text{ and } A_{\mathbf{c}, b^*} \text{ swapped } &\\ & {\mathsf{Biject}}_{\mathbf{c},b^*} = \{ \langle x, y \rangle ~ | ~ x \in A_{\mathbf{c}, b^*} \wedge y \in B_{\mathbf{c}, b^*} \wedge y \text{ is } \\ & <^B_{\mathbf{c}, b^*}\text{- maximal such that } R(x,y, \mathbf{c}) \} & \end{align*}$$

then:

• • $<^A_{\mathbf {c}, b^*}$ is a linear order on $A_{\mathbf {c}, b^*}$ and similarly for $<^B_{\mathbf {c}, b^*}$ ;

• • ${\mathsf {Biject}}_{\mathbf {c},b^*}$ is a bijection from $A_{\mathbf {c}, b*}$ to $B_{\mathbf {c}, b*}$ ;

• • for some b, $\{x \in A_{\mathbf {c}, b^*} ~ | R(x,b, \mathbf {c}) \}$ includes exactly one of any element of $A_{\mathbf {c}, b^*}$ and its $<^A_{\mathbf {c},b^*}$ successor, while including the first element and excluding the last element according to $<^A_{\mathbf {c}, b^*}$ .

The important points about $\phi $ are:

• • $\phi $ is in $L_P$ .

• • $\phi $ implies, over any embedded finite subset, that the cardinality of P is $0$ mod $4$ . This follows directly from the definitions and the assumption that $\mathbf {c} \cup \mathbf {b}^*$ has cardinality a multiple of $4$ . In particular this implication holds for subsets in $\mathcal C$ .

• • Conversely, for any $P_i \in \mathcal C$ , if $P_i$ has cardinality $0$ mod $4$ then $|A_i|$ and $|B_i|$ are even, and by choosing $\mathbf {c}$ and $b^*$ correctly—that is, as above—we see that $\phi $ holds.

But $\phi $ cannot be definable by a $1$ -RQC formula for embedded finite subsets in class $\mathcal C$ . Over this class, every L formula is equivalent (modulo expansion) to a formula using ${\mathsf {Biject}}_{\mathbf {c},b^*}$ , $<^A_{\mathbf {c}, b^*}$ , $<^B_{\mathbf {c}, b^*}$ for the correct $\mathbf {c}, b^*$ , where these are formulas in L extended with constants. So we have a formula in this language over a structure consisting of two linear ordered sets with a bijective correspondence. By a standard Ehrenfeucht–Fraïssé argument it is clear that the cardinality of the universe modulo $4$ cannot be defined in such a family. This completes the argument for Case 1.

Case 2: There exists some $b^*$ such that $\neg R(a_i,b^*, \mathbf {c})$ holds for all i, and for infinitely many $i R(b_i, b^*, \mathbf {c})$ .

This is argued symmetrically with Case 1 above.

Case 3. None of the above. If $R(b_i,b_j, \mathbf {c})$ and $\neg R(b_j,b_i, \mathbf {c})$ holds for all $i<j$ (or dually) then $R(x,y, \mathbf {c})$ defines an ordering on the $b_i$ . We let $\mathcal C$ be defined as above but without the $a_i$ or $b^*$ , and conclude analogously to Case 1.

Otherwise, by indiscernibility of $b_i$ over $\mathbf {c}$ , either $R(b_i,b_j, \mathbf {c})$ holds for all $i \neq j$ , or $\neg R(b_i,b_j, \mathbf {c})$ holds for all $i \neq j$ . Let us assume the latter. Extend the indiscernible sequence $\langle a_i,b_i \rangle _{i \in {\mathbb {N}}}$ by adding one more element $(a_\omega ,b_\omega )$ maximal in the ordering. So $R(a_i,b_\omega , \mathbf {c})$ holds for all $i \in {\mathbb {N}}$ , using indiscernibility, since $b_\omega $ is above $b_i$ for all standard i. But $\neg R(b_i,b_\omega , \mathbf {c})$ holds for all i, again by indiscernibility and $b_\omega $ being above $b_i \in {\mathbb {N}}$ . Hence we are in Case 1, a contradiction.

We now show that for $1$ -RQC, there is no difference between looking at embedded finite subsets and embedded finite models. That is, our weakening does not make any difference at the level of theories.

Proof. The interesting direction is assuming $1$ , Monadic-RQC and proving $1$ -RQC. Inductively it suffices to convert a formula $\phi $ of the form:

$$\begin{align*}\exists x ~ Q_1(u_1) \ldots Q_n(u_n) ~ \Gamma(x, \mathbf{u}, \mathbf{y}) \end{align*}$$

to an active domain formula, where the $Q_i$ are active domain quantifiers and $\Gamma $ is a Boolean combination of $\sigma $ formulas and L formulas. We can assume that x only occurs in L formulas. Let $\Gamma _L(x, \mathbf {u}, \mathbf {y})$ be the vector of L subformulas of $\Gamma $ . By Theorem 6.2, T is NIP. By [Reference Chernikov and Simon14] “local types are uniformly definable over finite sets”:

for every L formula $\phi (\mathbf {x}; \mathbf {y})$ , there is L formula $\delta (\mathbf {y}; \mathbf {p})$ such that: for any finite set S, for any $\mathbf {x}_0$ , there is a tuple $\mathbf {p}_0$ such that $\forall \mathbf {y} \in S ~ \phi (\mathbf {x}_0, \mathbf {y}) \leftrightarrow \delta (\mathbf {y}, \mathbf {p}_0)$ .

This means, in particular, that for $\gamma _i \in \Gamma _L$ there is $\delta _i(\mathbf {u}, \mathbf {p})$ such that for every finite set P for every $x_0, \mathbf {y}_0$ in a model of T there is $\mathbf {p}^i$ in P such that

$$\begin{align*}\forall \mathbf{u} \in P ~ \gamma_i(x_0, \mathbf{y}_0, \mathbf{u}) \leftrightarrow \delta_i(\mathbf{u}, \mathbf{p}^i). \end{align*}$$

This holds over all finite P, so in particular for the active domain of an embedded finite model.

We can thus replace $\phi $ with

$$ \begin{align*} \exists \mathbf{p}^1 \in {\mathsf{Adom}} \dots \exists \mathbf{p}^n \in {\mathsf{Adom}} \\\mathbf{p}^1 \ldots \mathbf{p}^n \text{ are definers for some } x \text{ for } \mathbf{y}\ \wedge \\~ Q_1(u_1) \ldots Q_n(u_n) ~ \Gamma'(\mathbf{u}, \mathbf{y}, \mathbf{p}^1 \ldots \mathbf{p}^n). \end{align*} $$

Here $\Gamma '$ is obtained by $\Gamma $ via replacing $\gamma _i$ with the corresponding $\delta _i$ . The first conjunct inside the existential has the obvious meaning, that the iff above holds for some x in the role of $x_0$ , with $\mathbf {y}$ in the role of $\mathbf {y}_0$ .

The first conjunct does not mention the additional relational signature $\sigma $ , and thus can be transformed into a $1$ -RQ formula via $1$ , Monadic-RQC.

Embedded subsets vs embedded models for isomorphism-invariant sentences: We contrast Theorem 6.3 with the situation when we restrict to isomorphism-invariant sentences. Recall that this denotes sentences of $L_V$ whose truth value in an embedded finite model for theory T depends only on the isomorphism type of the interpretations of the V predicates. Isomorphism-invariant $L_V$ sentences are ones that represent “pure relational computation.” Consider the random graph. The theory is not NIP, so by Theorem 6.2 it is not $1$ -RQC, and indeed not even $1$ , Monadic-RQC. And we have mentioned before that it is $2$ -RQC. In fact, every $L_V$ sentence can be converted to one that uses only Monadic second-order quantification over the active domain of the V predicates. Thus, informally, first-order quantification over the model gives you the power of Monadic second-order active domain quantification over the relational signature V.

We can also see that there are isomorphism-invariant $L \cup \{G(x,y)\}$ sentences $\gamma $ that are not equivalent to $1$ -RQ ones: by using unrestricted existential quantification to quantify over subsets of the nodes of G, we can express that G has a non-trivial connected component.

But consider an isomorphism-invariant sentence in $L_P$ . By the observation above, applicable to arbitrary $L_P$ sentences, these can all be expressed in restricted-quantifier Monadic second-order logic $\phi '$ , equivalent over embedded finite subsets to $\phi $ quantifying only over unary predicate P. By isomorphism-invariance, such a sentence is determined by its truth value on P lying within an order-indiscernible set, and thus we can rewrite such a sentence to $\phi "$ that does not mention the graph predicate of L at all: only equality atoms in it. Such a sentence can only define the set of finite P’s whose cardinality is in a finite or co-finite set. And such a set of finite P’s can also be defined in first-order logic over P. Thus there can be a difference between considering unary signatures and higher-arity signatures for isomorphism-invariant sentences.

7 Discussion and open issues

The main goal of this work was to revisit the evaluation of formulas that refer to a vocabulary $L_V$ mixing the signature of an infinite structure with a finite uninterpreted relational vocabulary. When this setting was studied in the 90s and the early 2000s, the emphasis was on the “very sunny day” scenario of RQC, where the expressiveness and evaluation complexity is not so different—from the point of view of “data complexity”—from that of evaluation of first-order formulas in a relational vocabulary over finite models.

In this article we show a much richer landscape, including:

• • Structures/theories where first-order logic over $L_V$ captures higher-order logic, or beyond. Thus both the expressiveness and complexity is high.

• • Structures/theories where first-order logic over $L_V$ does not correspond to any reasonable logic over the finite vocabulary, where this can be “fixed” by expanding the signature, and cases where it cannot be fixed.

We also locate the theory of finite and pseudo-finite fields lives within this landscape, showing that it is extremely expressive.

In each of the approaches to exploring evaluation on wider classes of infinite structures, we have left open many basic questions.

In terms of higher-order collapse, although we have separated $k+2$ from k-RQC for every k, we do not have the corresponding separation for $k+1$ and k. Thus it is possible that, for example, two consecutive levels of the hierarchy collapse, but the remainder are separated. Our construction from Section 3.1 shows that this would follow from the corresponding separation of pure spectra in finite model theory. We believe one can construct examples $T_k$ where the $L_P$ theory is bi-interpretable with k-higher-order logic over P, which would provide an equivalence between separations of RQC levels and expressiveness separations in higher-order logic.

On the issue of persistent unrestrictedness, one major question is whether any model theoretic criteria can enforce potential RQC. In particular, we do not know if NIP theories can always be expanded to become RQC. It is known that there are NIP theories (even stable theories) that cannot be expanded to fall into the known existing classes that imply $1$ -RQC [Reference Hieronymi, Nell and Walsberg24]. We have shown that Atomless Boolean Algebras are persistently unrestricted. We suspect the same argument applies to another well-known decidable theory, Büchi arithmetic, but we have not verified this.

Our discussion of algebraic examples focused on finite and pseudo-finite fields, and (in Appendix G) on the related topic of vector spaces with a bilinear form into a finite field. Even for pseudo-finite fields and finite fields, we do not have results when we restrict to characteristic $0$ .

On the issue of impact of the signature, the main question is whether k-RQC for $V=\{P\}$ , P a unary predicate P implies that k-RQC for general V.

The results on finite and pseudo-finite fields relate to the broader question of the connection between “tame definability properties” of a structure, such as k-RQC and the complexity of the underlying theory. In Proposition 3.11, we noted one simple connection: roughly speaking, high expressiveness of isomorphism-invariant $L_V$ sentences implies lower bounds for deciding L formulas. In Section 5 we showed that we can sometimes convert high expressiveness of isomorphism-invariant sentences into lower bounds on quantifier elimination in the theory, even when the number of variables is not fixed.

We know that, in general, properties like k-RQC do not even imply decidability of the theory, much less complexity bounds. This follows from the fact that purely model-theoretic properties—e.g., NFCP and o-minimality (see Section 2)—with no effectiveness requirement suffice to get $1$ -RQC. Conversely, Example 2 can be used to show that theories with modest complexity of satisfiability (even for fixed number of variables) may not even be $\omega $ -RQC. Intuitively, a Turing Machine can decide the theory by accessing additional parts of the structure, while restricted-quantifier formulas have no means to access this structure. One might hope that adding function symbols to the signature might close this gap, as it does in Example 4. But Theorem 4.5 shows that there are examples where this tactic cannot help.