2.1 An example: pure $\lambda$ -calculus

Consider the syntax of pure $\lambda$ -calculus extended with pattern metavariables. We list the Agda code in Figure 1, together with a corresponding presentation as inductive rules generating the syntax. We write $\Gamma;n\vdash t$ to mean t is a well-formed $\lambda$ -term in the context $\Gamma;n$ , consisting of two parts:

1. 1. a metavariable context (or metacontext) $\Gamma$ , which is either a formal error context $\bot$ , or a proper context, as a list $(M_{1}:m_{1},\dots,M_{p}:m_{p})$ , of metavariable declarations specifying metavariable symbols $M_{i}$ together with their arities, that is, their number of arguments $m_{i}$ ;

2. 2. a scope, which is a mere natural number indicating the highest possible free variable.

Fig. 1. Syntax of $\lambda$ -calculus (Section 2.1)

We use the bold face $\boldsymbol{\Gamma}$ for any proper metacontext. In the Agda code, we adopt a nameless encoding of proper metacontexts: they are mere lists of metavariable arities, and metavariables are referred to by their index in the list. The type of metacontexts is formally defined as , where is an inductive type with an error constructor $\bot$ and a proper constructor $\lfloor-\rfloor$ taking as argument an element of type X. Therefore, $\boldsymbol{\Gamma}$ typically translates into $\lfloor\Gamma\rfloor$ in the implementation. To alleviate notations, we also adopt a dotted convention in Agda to mean that a proper metacontext is involved. For example, and are, respectively, defined as and .

Free variables are indexed from 1, and we use the de Bruijn level convention: the variable bound in $\boldsymbol{\Gamma};n\vdash\lambda t$ is $n+1$ , not 0, as it would be using de Bruijn indices (De Bruijn, Reference De Bruijn1972). In Agda, variables in the scope n consist of elements of , the type of natural numbers betweenFootnote ³ 1 and n.

The last term constructor $\mbox{!}$ builds a well-formed term in any error context $\bot;n$ . We call it an error term: it is the only one available in such contexts. Proper terms, that is, terms well-formed in a proper metacontext, are built from application, $\lambda$ -abstraction and variables: they generate the (proper) syntax of $\lambda$ -calculus. Note that $\mbox{!}$ cannot occur as a sub-term of a proper term.

The names of constructors of $\lambda$ -calculus for application, $\lambda$ -abstraction, and variables are dotted to indicate that they are only available in a proper metacontext. “Improper” versions of those, defined in any metacontext, are also implemented in the obvious way, coinciding with the constructors in a proper context, or returning $\mbox{!}$ in the error context.

Let us focus on the penultimate constructor, building a metavariable application in the context $\boldsymbol{\Gamma};n$ . The argument of type $m\in\boldsymbol{\Gamma}$ is an index of any element m in the list $\boldsymbol{\Gamma}$ . In the pattern fragment, a metavariable of arity m can be applied to a list of size m consisting of distinct variables in the scope n, that is, natural numbers between 1 and n. We denote by $\hom(m,n)$ this set of lists. To make the Agda implementation easier, we did not enforce the uniqueness restriction in the definition of . However, our unification algorithm is guaranteed to produce correct outputs only if this constraint is satisfied in the inputs.

The Agda implementation of metavariable substitutions for $\lambda$ -calculus is listed in the first box of Figure 2. We call a substitution proper if the domain is proper, successful if the target is also proper. Note that a substitution is successful if and only if the target is proper, because there is only one metavariable substitution $1_{\bot}$ from the error context: it is the formal identity substitution, targeting itself. A metavariable substitution $\sigma:\boldsymbol{\Gamma}\rightarrow\Delta$ from a proper context assigns to each metavariable M of arity m in $\boldsymbol{\Gamma}$ a term $\Delta;m\vdash\sigma_{M}$ .

Fig. 2. Metavariable substitution for $\lambda$ -calculus (Section 2.1)

This assignment extends (through a recursive definition) to any term $\boldsymbol{\Gamma};n\vdash t$ , yielding a term $\Delta;n\vdash t[\sigma]$ . The congruence cases involve improper versions of the operations, as the target metacontext may not be proper. The base case is $M(x_{1},\dots,x_{m})[\sigma]=\sigma_{M}\{x\},$ where $-\{x\}$ is variable renaming, defined by recursion: it replaces each variable i by $x_{i}$ . Renaming a $\lambda$ -abstraction requires extending the renaming to to take into account the additional bound variable $\underline{p+1}$ , which is renamed to $\underline{q+1}$ . Then, $(\lambda t)\{x\}$ is defined as $\lambda(t\{x\uparrow\})$ . While metavariable substitutions change the metacontext of the substituted term, renamings change the scope.

The identity substitution $1_{\boldsymbol{\Gamma}}:\boldsymbol{\Gamma}\rightarrow\boldsymbol{\Gamma}$ is defined by the term $M(1,\dots,m)$ for each metavariable declaration $M:m\in\boldsymbol{\Gamma}$ . The composition $\delta[\sigma]:\boldsymbol{\Gamma_{1}}\rightarrow\Gamma_{3}$ of two substitutions $\delta:\boldsymbol{\Gamma_{1}}\rightarrow\Gamma_{2}$ and $\sigma:\Gamma_{2}\rightarrow\Gamma_{3}$ is defined as $M\mapsto\delta_{M}[\sigma]$ .

We write $\Gamma\vdash t=u\Rightarrow\sigma\dashv\Delta$ to mean that $\sigma$ is a most general unifier (mgu) of t and u, as in Section 1. More explicitly, a unifier of two terms $\Gamma;n\vdash t,u$ is a substitution $\sigma:\Gamma\rightarrow\Delta$ such that $t[\sigma]=u[\sigma]$ . We call it successful if the underlying substitution is. A mgu $\sigma:\Gamma\rightarrow\Delta$ of t and u is a unifier that uniquely factors any other unifier $\delta:\Gamma\rightarrow\Delta'$ , in the sense that there exists a unique $\delta':\Delta\rightarrow\Delta'$ such that $\delta=\sigma[\delta']$ .

In the notation $\Gamma\vdash t=u\Rightarrow\sigma\dashv\Delta$ , the symbol $\Rightarrow$ separates the input and the output of the unification algorithm. Indeed, as can be seen in Figure 3, the function takes two terms $\Gamma;n\vdash t,u$ as input and returns a record with two fields: a context $\Delta$ , which is $\bot$ in case there is no successful unifier, and a substitution $\sigma:\Gamma\rightarrow\Delta$ , which is a mgu of t and u (the mgu property is however not explicitly enforced by the type signature).

Fig. 3. Unification for $\lambda$ -calculus

This unification function recursively inspects the structure of the given terms until reaching a metavariable at the top level, as seen in the box of Figure 3. The last two cases handle unification of two error terms, and unification of two different rigid term constructors (application, $\lambda$ -abstraction, or variables), resulting in failure.

When reaching a metavariable application M(x) at the top level of either term in a metacontext $\boldsymbol{\Gamma}$ , denoting by t the other term, three situations are considered by the auxiliary function :

1. 1. t is a metavariable application M(y);

2. 2. t is not a metavariable application and M occurs deeply in t;

3. 3. M does not occur in t.

The function returns in the first case, in the second case, and in the last case, where t’ is t but considered in the context $\boldsymbol{\Gamma}$ without M, denoted by $\boldsymbol{\Gamma}\backslash M$ . In the first case, the line computes the vector of common positions Footnote ⁴ of x and y, that is, the maximal vector of (distinct) positions $(z_{1},\dots,z_{p})$ such that $x_{\vec{z}}=y_{\vec{z}}$ . We denoteFootnote ⁵ such a situation by . The most general unifier $\sigma$ coincides with the identity substitution except that the declaration $M:m$ is replaced by a metavariable declaration $P:p$ in the context $\boldsymbol{\Gamma}$ , and $\sigma$ maps M to P(z).

Recall that in the Agda implementation, metavariables are natural numbers referring to positions in the metacontext, which is just a list of arities. In the Agda code, $\Gamma[M:p]$ denotes the metacontext $\Gamma$ where the $M^{th}$ element has been replaced with p. Furthermore, $(M:p)(u')$ denotes the metavariable M applied to y’, where the annotation $:p$ is an explicit coercion making M a metavariable of $\Gamma[M:p]$ rather than of $\Gamma$ . Finally, the output metasubstitution $\sigma$ between $\Gamma$ and $\Gamma[M:p]$ , denoted by $M\mapsto-(x)$ in the code, coincide with the identity substitution of $\Gamma$ , except for $\sigma_{M}$ which is defined as M(x).

The second case tackles unification of a metavariable application with a term in which the metavariable occurs deeply. It is handled by the failing rule : there is no (successful) unifier because the size of both hand sides can never match after substitution.

The last case described by the rule is unification of M(x) with a term t in which M does not occur. This kind of unification problem is handled specifically by a previously defined function , listed in Figure 4. The intuition is that M(x) and t should be unified by replacing M with $t[x_{i}\mapsto i]$ . However, this only makes sense if the free variables of t are in x. For example, if t is an outbound variable, that is, a variable that does not occur in x, then there is no unifier. Nonetheless, it is possible to prune the outbound variables in t as long as they only occur in metavariable arguments, by restricting the arities of those metavariables. As an example, if t is a metavariable application N(x,y), then although the free variables are not all included in x, the most general unifier still exists, essentially replacing N with M, discarding the outbound variable y.

Fig. 4. Pruning for $\lambda$ -calculus

The pruning phase runs in the metacontext with M removed. We use the notation $\Gamma\vdash_{}t\boldsymbol{:>}x\Rightarrow t';\sigma\dashv\Delta$ , where t is a term in the metacontext $\Gamma$ , while x is the argument of the metavariable whose arity m is left implicit, as well as its (irrelevant) name. The output is a metacontext $\Delta$ , together with a term t’ in context $\Delta;m$ , and a substitution $\sigma:\Gamma\rightarrow\Delta$ . If $\Gamma$ is proper, this is precisely the data for the most general unifier of t and M(x), considered in the extended metacontext $M:m,\Gamma$ . Following the above pruning intuition, t’ is the term t where the outbound variables have been pruned, in case of success. This justifies the type signature of .

The function recursively inspects its argument. The base metavariable case corresponds to unification of M(x) and M’(y) where M and M’ are distinct metavariables. In this case, the line computes the vectors of common value positions $(x_{1}',\dots,x_{p}')$ and $(y_{1}',\dots,y'_{p})$ between $x_{1},\dots,x_{m}$ and $y_{1},\dots,y_{m'}$ , that is, the pair of maximal lists $(\vec{x'},\vec{y'})$ of distinct positions such that $x_{\vec{x'}}=y_{\vec{y'}}$ . We denoteFootnote ⁶ such a situation by . The most general unifier $\sigma$ coincides with the identity substitution except that the metavariables M and M’ are removed from the context and replaced by a single metavariable declaration $P:p$ . Then, $\sigma$ maps M to P(x’) and M’ to P(y’).

As for the rule , the implementation merely replaces the metavariable arity of M with p, using the same Agda notations as in the auxiliary function in Figure 3.

The intuition for the application case is that if we want to unify M(x) with $t\ u$ , we can refine M(x) to be $M_{1}(x)\ M_{2}(x)$ , where $M_{1}$ and $M_{2}$ are two fresh metavariables to be unified with t and u. We can process those unification problems in order. If the first one yields t’ and the substitution $\sigma_{1}$ , and then we refine the second unification problem by replacing u with $u[\sigma_{1}]$ . Assuming the output of this second unification is u’ and $\sigma_{2}$ , then M should be replaced accordingly with $t'[\sigma_{2}]\ u'$ . Note that this really involves improper application, taking into account the following three subcases at once.

\[\dfrac{\begin{array}{c}\boldsymbol{\Gamma}\vdash_{}t\boldsymbol{:>}x\Rightarrow t';\sigma_{1}\dashv\boldsymbol{\Delta_{1}}\\\boldsymbol{\Delta_{1}}\vdash_{}u[\sigma_{1}]\boldsymbol{:>}x\Rightarrow u';\sigma_{2}\dashv\boldsymbol{\Delta_{2}}\end{array}}{\boldsymbol{\Gamma}\vdash_{}t\ u\boldsymbol{:>}x\Rightarrow t'[\sigma_{2}]\ u';\sigma_{1}[\sigma_{2}]\dashv\boldsymbol{\Delta_{2}}}\]

\[\dfrac{\begin{array}{c}\boldsymbol{\Gamma}\vdash_{}t\boldsymbol{:>}x\Rightarrow t';\sigma_{1}\dashv\boldsymbol{\Delta_{1}}\\\boldsymbol{\Delta_{1}}\vdash_{}u[\sigma_{1}]\boldsymbol{:>}x\Rightarrow\mbox{!};\mbox{!}_{s}\dashv\bot\end{array}}{\boldsymbol{\Gamma}\vdash_{}t\ u\boldsymbol{:>}x\Rightarrow\mbox{!};\mbox{!}_{s}\dashv\bot}\quad\dfrac{\begin{array}{c}\boldsymbol{\Gamma}\vdash_{}t\boldsymbol{:>}x\Rightarrow\mbox{!};\mbox{!}_{s}\dashv\bot\\\bot\vdash_{}\mbox{!}\boldsymbol{:>}x\Rightarrow\mbox{!};\mbox{!}_{s}\dashv\bot\end{array}}{\boldsymbol{\Gamma}\vdash_{}t\ u\boldsymbol{:>}x\Rightarrow\mbox{!};\mbox{!}_{s}\dashv\bot}\]

The same intuition applies for $\lambda$ -abstraction, but here we apply the fresh metavariable corresponding to the body of the $\lambda$ -abstraction to the bound variable $n+1$ , which needs not be pruned. In the variable case, $i\{x\}^{-1}$ returns the index j such that $i=x_{j}$ , or fails if no such j exist.

This ends our description of the unification algorithm, in the specific case of pure $\lambda$ -calculus.

2.2 Generalisation

In this section, we show how to abstract over $\lambda$ -calculus to get a generic algorithm for pattern unification, parameterised by our new notion of specification to account for syntax with metavariables. We split this notion in two parts:

1. 1. a notion of generalised binding signature, or GB-signature (formally introduced in Definition 23), specifying a syntax with metavariables, for which unification problems can be stated;

2. 2. some additional structures used in the algorithm to solve those unification problems, as well as properties ensuring its correctness, making the GB-signature pattern-friendly (see Definition 25).

This separation is motivated by the fact that in the case of $\lambda$ -calculus, the vectors of common (value) positions are involved in the algorithm, but not in the definition of the syntax and associated operations (renaming, metavariable substitution).

A GB-signature consists in a tuple consisting of

• • a small category $\mathcal{A}$ whose objects are called arities or scopes, and whose morphisms are called patterns or renamings;

• • for each variable context a, a set of operation symbols O(a);

• • for each operation symbol $o\in O(a)$ , a list of scopes $\alpha_{o}=(\overline{o}_{1},\dots,\overline{o}_{n})$ .

such that O and are functorial in a suitable sense. In particular, given a morphism $x\colon a\rightarrow b$ in $\mathcal{A}$ and operation symbol $o\in O(a)$ with $\alpha_{o}=(\overline{o}_{1},\dots,\overline{o}_{n})$ , there is

• • an operation symbol $o\{x\}$ in O(b);

• • a vector $x^{o}:\alpha_{o}\dashrightarrow\alpha_{o\{x\}}$ of renamings, that is, a list $(x_{1}^{o},\dots,x_{n}^{o})$ of morphisms in $\mathcal{A}$ such that $x_{i}^{o}\colon\overline{o}_{i}\rightarrow\overline{o\{x\}}_{i}$ .

Functoriality ensures that the generated syntax supports renaming: given a morphism $x:a\rightarrow b$ in $\mathcal{A}$ and a term $\Gamma;a\vdash t$ , we recursively define $\Gamma;b\vdash t\{x\}$ by $M(x\circ y)$ if $t=M(y)$ , or by if .

The Agda implementation in Figure 5 does not include properties such as associativity of morphism composition, although they are assumed in the proof of correctness. For example, the latter associativity property ensures that composition of metavariable substitutions is associative.

Fig. 5. Generalised binding signatures in Agda

Example 6. We give the signature for pure $\lambda$ -calculus. As explained in the introduction, we take $\mathcal{A}=\mathbb{F}_{m}$ . In the scope n, we have n nullary available operation symbols (one for each variable), one unary operation $abs^{n}$ , and one binary operation $app^{n}$ , so that $O(n)=\{1,\dots n,abs^{n},app^{n}\}$ , with associated arities $\alpha_{i}=()$ , $\alpha_{abs^{n}}=(n+1)$ and $\alpha_{app^{n}}=(n,n)$ . The corresponding Agda implementation can be found in Figure 6.

Fig. 6. Implementation of the signature of pure $\lambda$ -calculus

The syntax specified by a GB-signature is inductively defined in Figure 7, where a context $\Gamma;a$ is defined as in Section 2.1 for $\lambda$ -calculus, except that scopes and metavariable types are objects of $\mathcal{A}$ instead of natural numbers.

Fig. 7. Syntax generated by a GB-signature

We call a term rigid if it is of the shape $o(\dots)$ , flexible if it is some metavariable application $M(\dots)$ .

Recall that the Agda code uses a nameless convention for metacontexts: they are just lists of scopes. Therefore, the arity $\alpha_{o}$ of an operation o can be considered as a metacontext. It follows that the argument of an operation o in the context $\boldsymbol{\Gamma};a$ can be specified either as a metavariable substitution from $\alpha_{o}=(\overline{o}_{1},\dots,\overline{o}_{n})$ to $\boldsymbol{\Gamma}$ , as in the Agda code, or explicitly as a list of terms $(t_{1},\dots,t_{n})$ such that $\boldsymbol{\Gamma};\overline{o}_{i}\vdash t_{i}$ , as in the rule . In the following, we will use either interpretation.

Since metasubstitutions occur in the syntax of terms, we define substitution on terms mutually with composition of metasubstitutions in Figure 8. We are similarly led to mutually define unification of terms and unification of metasubstitutions. Given two substitutions $\delta_{1},\delta_{2}:\Gamma'\rightarrow\Gamma$ , we write $\Gamma\vdash\delta_{1}=\delta_{2}\Rightarrow\sigma\dashv\Delta$ to mean that $\sigma:\Gamma\rightarrow\Delta$ unifies $\delta_{1}$ and $\delta_{2}$ , in the sense that $\delta_{1}[\sigma]=\delta_{2}[\sigma]$ , and is the most general one, that is, it uniquely factors any other unifier of $\delta_{1}$ and $\delta_{2}$ . In Figure 9, we list the type signatures of the functions involved in the algorithm: the main unification function is split in two functions, for single terms, and for substitutions. Similarly, we define pruning of terms mutually with pruning of proper substitutions, using the below notion of vector renaming for metacontexts.

Fig. 8. Metavariable substitution for a GB-signature (Section 2.2)

Fig. 9. Type signatures of unification and pruning

This is is compatible with the notion of vector of renamings that we introduced at the beginning of the section, if we take metacontexts to be mere lists of scopes, as in the Agda implementation.

Given a substitution $\delta:\boldsymbol{\Gamma'}\rightarrow\Gamma$ and a vector $x:\boldsymbol{\Gamma''}\dashrightarrow\boldsymbol{\Gamma'}$ of renamings, the judgement $\Gamma\vdash_{}\delta\boldsymbol{:>}x\Rightarrow\delta';\sigma\dashv\Delta$ means that the substitution $\sigma:\Gamma\rightarrow\Delta$ extended with $\delta':\boldsymbol{\Gamma''}\rightarrow\Delta$ is the most general unifier of $\delta$ and $\overline{x}$ as substitutions from $\boldsymbol{\Gamma''},\Gamma$ to $\Delta$ , where $\boldsymbol{\Gamma''},\Gamma$ is the concatenation of contexts if $\Gamma$ is proper, or $\bot$ otherwise.

The unification algorithm is summarised in Figure 10, next to the $\lambda$ -calculus implementation (Figure 11) for comparison. In that special case, unification of two metavariable applications requires computing the vector of common positions or value positions of their arguments, depending on whether the involved metavariables are identical. Both vectors are characterised as equalisers or pullbacks in the category of natural numbers and injective renamings between them, thus providing a canonical replacement in the generic algorithm, along with new interpretations of the notations and as equalisers and pullbacks.

Fig. 10. Our generic pattern unification algorithm

Fig. 11. Pattern unification for $\lambda$ -calculus (Section 2.1)

Notation 9. We write and to respectively denote an equaliser and pullback in $\mathcal{A}$ as below.

Let us now comment on pruning rigid terms, when we want to unify an operation $o(\delta)$ with a fresh metavariable application M(x). Any unifier must replace M with an operation $o'(\delta')$ , such that $o'\{x\}(\delta'\{x^{o'}\})=o(\delta)$ , so that, in particular, $o'\{x\}=o$ . In other words, o must have a preimage o’ for renaming by x. This is precisely the point of the inverse renaming $o\{x\}^{-1}$ in the Agda code: it returns a preimage o’ if it exists, or fails. In the $\lambda$ -calculus case, this check is only explicit for variables, since there is a single version of application and $\lambda$ -abstraction symbols in any variable context. The algorithm relies on GB-signatures with additional components listed in Figure 12. We call such GB-signatures binding-friendly. To sum up, equalisers and pullbacks are used when unifying two metavariable applications; equality of operation symbols is used when unifying two rigid terms; inverse renaming is used when pruning a rigid term.

Fig. 12. Pattern-friendly GB-signatures in Agda

The formal notion of pattern-friendly signatures (Definition 25) includes additional properties ensuring that the algorithm is correct. One consequence of those properties is that inverse renaming is unique: there is at most one preimage by renaming.

3.1. Pattern unification as a coequaliser construction

In this section, we assume given a GB-signature S and explain how most general unifiers can be thought of as equalisers in a multi-sorted Lawvere theory, as is well-known in the first-order case (Rydeheard & Burstall, Reference Rydeheard and Burstall1988; Barr & Wells, Reference Barr and Wells1990). We furthermore provide a formal justification for the error metacontext $\bot$ .

This relies on functoriality of GB-signatures that we will spell out formally in the next section. There, we will see in Proposition 33 that this category fully faithfully embeds in a Kleisli category for a monad generated by S on $[\mathcal{A},\mathrm{Set}]$ .

This motivates a new interpretation of the unification notation, that we introduce later in Notation 20, after explaining how failure is handled categorically. Indeed, pattern unification is typically stated as the existence of a coequaliser on the condition that there is a unifier in this category $\mathrm{MCon}(S)$ . But we can get rid of this condition by considering the category $\mathrm{MCon}(S)$ freely extended with a terminal object $\bot$ , resulting in the full category of metacontexts and substitutions.

Notation 14 We denote by $\mbox{!}_{s}$ any terminal morphism to $\bot$ in $\mathscr{B}_{\bot}$ .

In Section 2.1, we already made sense of this extension. Let us rephrase our explanations from a categorical perspective. Adding a terminal object results in adding a terminal cocone to all diagrams. As a consequence, we have the following lemma.

1. 1. J has a colimit as long as there exists a cocone;

2. 2. J has a colimit in $\mathscr{B}_{\bot}$ .

The following results are also useful.

1. (i) The canonical embedding functor $\mathscr{B}\rightarrow\mathscr{B}_{\bot}$ preserves colimits.

2. (ii) Any diagram J in $\mathscr{B}_{\bot}$ such that $\bot$ is in its image has a colimit given by the terminal cocone on $\bot$ .

Accordingly, one can define informally “concatenation” of a metacontext with $\bot$ as $\bot$ . The main property of this extension for our purposes is the following corollary.

This justifies the following interpretation to the unification notation.

Notation 20. $\Gamma\vdash\delta_{1}=\delta_{2}\Rightarrow\sigma\dashv\Delta$ denotes a coequaliser in $\mathrm{MCon}(S)_{\bot}$ .

Categorically speaking, our pattern unification algorithm provides an explicit proof of the following statement, where the conditions for a signature to be pattern-friendly are introduced in the next section (Definition 25).

3.2 Initial algebra semantics for GB-signatures

The proofs of various statements presented in this section are detailed in Section 3.3.

• • a small category $\mathcal{A}$ of arities and renamings between them;

• • a functor $\mathcal{O}_{-}(-):\mathbb{N}\times\mathcal{A}\rightarrow\mathrm{Set}$ of operation symbols;

• • a family of functors $(\alpha_{n,i}:\int\mathcal{O}_{n}\rightarrow\mathcal{A})_{n,i\leq n}$ indexed by natural numbers i,n such that $1\leq i\leq n$ ,

  where $\int\mathcal{O}_{n}$ denotes the category of elements of $\mathcal{O}_{n}:\mathcal{A}\rightarrow\mathrm{Set}$ , defined as follows:

• • objects are pairs (a,o) such that $o\in\mathcal{O}_{n}(a)$

• • a morphism between (a,o) and (a’,o’) is a morphism $f:a\rightarrow a'$ such that $o\{f\}=o'$ where $o\{f\}$ denotes the image of o by the function $\mathcal{O}_{n}(f):\mathcal{O}_{n}(a)\rightarrow\mathcal{O}_{n}(a')$ .

Notation 24. Given a GB-signature and $o\in\mathcal{O}_{n}(a)$ , we write $\overline{o}_{j}$ for $\alpha_{n,j}(o)$ and $\alpha_{o}$ for the tuple $(\overline{o}_{1},\dots,\overline{o}_{n})$ .

We now introduce our conditions for the generic unification algorithm to be correct.

1. 1. $\mathcal{A}$ has finite connected limits (or equivalently, $\mathcal{A}$ has pullbacks and equalisers);

2. 2. all morphisms in $\mathcal{A}$ are monomorphic;

3. 3. each $\mathcal{O}_{n}(-):\mathcal{A}\rightarrow\mathrm{Set}$ preserves finite connected limits;

4. 4. each $\alpha_{n,i}$ preserves finite connected limits.

Note that $M(f)\stackrel{?}{=}*$ has two unifiers given by the two operations in scope a, but none of them factors the other and so there is no most general unifier.

These conditions ensure the following two properties.

Property 27 (proved in Section 3.3.1). The following properties hold for pattern-friendly signatures.

1. (i) The action of $\mathcal{O}_{n}:\mathcal{A}\rightarrow\mathrm{Set}$ on any renaming is an injection: given any $o\in\mathcal{O}_{n}(b)$ and renaming $f:a\rightarrow b$ , there is at most one $o'\in\mathcal{O}_{n}(a)$ such that $o=o'\{f\}$ .

2. (ii) Let $\mathcal{L}$ be the functor $\mathcal{A}^{op}\xrightarrow{}\mathrm{MCon}(S)_{\bot}$ mapping a morphism $x\in\hom_{\mathcal{A}}(b,a)$ to the substitution from $(X:a)$ to $(X:b)$ consisting of the single term X(x). Then, $\mathcal{L}$ preserves finite connected colimits: it maps pullbacks and equalisers in $\mathcal{A}$ to pushouts and coequalisers in $\mathrm{MCon}(S)_{\bot}$ .

The first property is used for soundness of the rules and . It is not satisfied in the counter-example of Remark 26. The second one is used to justify unification of two metavariables applications as pullbacks and equalisers in $\mathcal{A}$ , in the rules and .

In the rest of this section, we provide Initial Algebra Semantics for the generated syntax (this is used in the proof of Assumption 27.(ii)).

Any GB-signature , generates an endofunctor $F_{S}$ on $[\mathcal{A},\mathrm{Set}]$ , that we denote by just F when the context is clear, defined by

\[F_{S}(X)_{a}=\coprod_{n\in\mathbb{N}}\coprod_{o\in\mathcal{O}_{n}(a)}X_{\overline{o}_{1}}\times\dots\times X_{\overline{o}_{n}}.\]

Lemma 30. The proper syntax generated by a GB-signature (see Figure 7) is recovered as free algebras for F. More precisely, given a metacontext $\boldsymbol{\Gamma}=(M_{1}:m_{1},\dots,M_{p}:m_{p})$ ,

\[T(\underline{\Gamma})_{a}\cong\{t\ |\ \Gamma;a\vdash t\}\]

where $\underline{\boldsymbol{\Gamma}}:\mathcal{A}\rightarrow\mathrm{Set}$ is defined as the coproduct of representable functors mapping a to $\coprod_{i}\hom_{\mathcal{A}}(m_{i},a)$ . Moreover, the action of $T(\underline{\boldsymbol{\Gamma}})$ on morphisms of $\mathcal{A}$ correspond to renaming.

Notation 31. Given a proper metacontext $\boldsymbol{\Gamma}$ . We sometimes denote $\underline{\boldsymbol{\Gamma}}$ just by $\boldsymbol{\Gamma}$ .

If $\boldsymbol{\Gamma}=(M_{1}:m_{1},...,M_{p}:m_{p})$ and $\boldsymbol{\Delta}$ are metacontexts, a Kleisli morphism $\sigma:\boldsymbol{\Gamma}\rightarrow T\boldsymbol{\Delta}$ is equivalently given (by combining the above lemma, the Yoneda Lemma, and the universal property of coproducts) by a metavariable substitution from $\boldsymbol{\Gamma}$ to $\boldsymbol{\Delta}$ . Moreover, Kleisli composition corresponds to composition of substitutions. This provides a formal link between the category of metacontexts $\mathrm{MCon}(S)$ and the Kleisli category of T (see Section for a definition of the latter category).

Notation 32. We denote the Kleisli category of a monad T on $\mathscr{B}$ by $Kl_{T}$ : the objects are the same as those of $\mathscr{B}$ , and a Kleisli morphism from A to B is a morphism $A\rightarrow TB$ in $\mathscr{B}$ . We denote the Kleisli composition of $f:A\rightarrow TB$ and $g:B\rightarrow TC$ by $f[g]:A\rightarrow TC$ .

We exploit this characterisation to prove various properties of this category when the signature is pattern-friendly.

Notation 35. Given a GB-signature , we denote the full subcategory of $[\mathcal{A},\mathrm{Set}]$ consisting of functors preserving finite connected limits by $\mathscr{C}_{S}$ , or sometimes by $\mathscr{C}$ , leaving S implicit.

We now assume given a pattern-friendly signature .

3.3 Proofs of statements in Section 3.2

3.3.1 Property 27

We use the notations and definitions of Section 3.2.

Let us first prove the first item.

Proof of Property 27.(i) We show that given any $o\in\mathcal{O}_{n}(b)$ and renaming $f:a\rightarrow b$ , there is at most one $o'\in\mathcal{O}_{n}(a)$ such that $o=o'\{f\}$ .

Note that a morphism $f:a\rightarrow b$ is monomorphic if and only if the following square is a pullback (Mac Lane, Reference Mac Lane1998, Exercise III.4.4), as shown by unfolding the universal property of this particular limit.

Therefore, since $\mathcal{O}_{n}$ preserves finite connected limits, it also preserves monomorphisms.

The rest of this section is devoted to the proof of Property 27.(ii).

By right continuity of the homset bifunctor, any representable functor is in $\mathscr{C}$ and thus the embedding $\mathscr{C}\rightarrow[\mathcal{A},\mathrm{Set}]$ factors the Yoneda embedding $\mathcal{A}^{op}\rightarrow[\mathcal{A},\mathrm{Set}]$ .

Lemma 39. Let $\mathscr{D}$ denote the opposite category of $\mathcal{A}$ and $K:\mathscr{D}\rightarrow\mathscr{C}$ the factorisation of $\mathscr{C}\rightarrow[\mathcal{A},\mathrm{Set}]$ by the Yoneda embedding. Then, $K:\mathscr{D}\rightarrow\mathscr{C}$ preserves finite connected colimits.

Proof This essentially follows from the fact functors in $\mathscr{C}$ preserves finite connected limits. Let us detail the argument: let $y:\mathcal{A}^{op}\rightarrow[\mathcal{A},\mathrm{Set}]$ denote the Yoneda embedding and $J:\mathscr{C}\rightarrow[\mathcal{A},\mathrm{Set}]$ denote the canonical embedding, so that

(3.1) \begin{equation}y=J\circ K.\end{equation}

Now consider a finite connected limit $\lim F$ in $\mathcal{A}$ . Then,

These isomorphisms are natural in X and thus $K\lim F\cong\mathrm{colim}\ KF$ .

Proof of Assumption 27.(ii) Note that $\mathcal{L}$ factors as

\[\mathscr{D}\xrightarrow{\mathcal{L}^{\bullet}}\mathrm{MCon}(S)\hookrightarrow\mathrm{MCon}(S)_{\bot},\]

where the right embedding preserves colimits by Lemma 17.(i), so it is enough to show that $\mathcal{L}^{\bullet}$ preserves finite connected colimits. Let $T_{|\mathscr{C}}$ be the monad T restricted to $\mathscr{C}$ , following Corollary 38. Since $K:\mathscr{D}\rightarrow\mathscr{C}$ preserves finite connected colimits (Lemma 39), composing it with the left adjoint $\mathscr{C}\rightarrow Kl_{T_{|\mathscr{C}}}$ yields a functor $\mathscr{D}\rightarrow Kl_{T_{|\mathscr{C}}}$ also preserving those colimits. Since it factors as $\mathscr{D}\xrightarrow{\mathcal{L}^{\bullet}}\mathrm{MCon}(S)\hookrightarrow Kl_{T_{|\mathscr{C}}}$ , where the right functor is full and faithful, $\mathcal{L}^{\bullet}$ also preserves finite connected colimits.

3.3.3 Lemma 36

Notation 40. Given a functor $F:I\rightarrow\mathscr{B}$ , we denote the limit (resp. colimit) of F by $\int_{i:I}F(i)$ or $\lim F$ (resp. $\int^{i:I}F(i)$ or $\mathrm{colim}\ F$ ) and the canonical projection $\lim F\rightarrow Fi$ by $p_{i}$ for any object i of I.

This section is dedicated to the proof of the following lemma.

Lemma 41. Given a GB-signature such that $\mathcal{A}$ has finite connected limits, $F_{S}$ restricts as an endofunctor on the full subcategory $\mathscr{C}$ of $[\mathcal{A},\mathrm{Set}]$ consisting of functors preserving finite connected limits if and only if each $\mathcal{O}_{n}\in\mathscr{C}$ , and $\alpha_{n,i}:\int\mathcal{O}_{n}\rightarrow\mathcal{A}$ preserves finite connected limits.

We first introduce a bunch of intermediate lemmas.

Lemma 42. Let $F:\mathscr{B}\rightarrow\mathrm{Set}$ be a functor. For any functor $G:I\rightarrow\int F$ , denoting by H the composite functor $I\xrightarrow{G}\int F\rightarrow\mathscr{B}$ , there exists a unique $x\in\lim(F\circ H)$ such that $Gi=(Hi,p_{i}(x))$ .

Proof $\int F$ is isomorphic to the opposite of the comma category $y/F$ , where $y:\mathscr{B}^{op}\rightarrow[\mathscr{B},\mathrm{Set}]$ is the Yoneda embedding. The statement follows from the universal property of a comma category.

Lemma 43. Let $F:\mathscr{B}\rightarrow\mathrm{Set}$ and $G:I\rightarrow\int F$ such that F preserves the limit of $H:I\xrightarrow{G}\int F\xrightarrow{}\mathscr{B}$ . Then, there exists a unique $x\in F\lim H$ such that $Gi=(Hi,Fp_{i}(x))$ and moreover, $(\lim H,x)$ is the limit of G.

Proof The unique existence of $x\in F\lim H$ such that $Gi=(Hi,Fp_{i}(x))$ follows from Lemma 42 and the fact that F preserves $\lim H$ . Let $\mathscr{C}$ denote the full subcategory of $[\mathscr{B},\mathrm{Set}]$ of functors preserving $\lim G$ . Note that $\int F$ is isomorphic to the opposite of the comma category $K/F$ , where $K:\mathscr{B}^{op}\rightarrow\mathscr{C}$ is the Yoneda embedding, which preserves $\mathrm{colim}\ G$ , by an argument similar to the proof of Lemma 39. We conclude from the fact that the forgetful functor from a comma category $L/R$ to the product of the categories creates colimits that L preserve.

Corollary 44. Let I be a small category, $\mathscr{B}$ and $\mathscr{B}'$ be categories with I-limits (i.e., limits of any diagram over I). Let $F:\mathscr{B}\rightarrow\mathrm{Set}$ be a functor preserving those limits. Then, $\int F$ has I-limits, preserved by the projection $\int F\rightarrow\mathscr{B}$ . Moreover, a functor $G:\int F\rightarrow\mathscr{B}'$ preserves them if and only if for any $d:I\rightarrow\mathscr{B}$ and $x\in F\lim d$ , the canonical morphism $G(\lim d,x)\rightarrow\int_{i:I}G(d_{i},Fp_{i}(x))$ is an isomorphism.

Proof By Lemma 43, a diagram $d':I\rightarrow\int F$ is equivalently given by $d:I\rightarrow\mathscr{B}$ and $x\in F\lim d$ , recovering d’ as $d'_{i}=(d_{i},Fp_{i}(x))$ , and moreover $\lim d'=(\lim d,x)$ .

Corollary 45. Assuming that $\mathcal{A}$ has finite connected limits and each $\mathcal{O}_{n}$ preserves finite connected limits, the finite limit preservation on $\alpha:\int J\rightarrow\mathcal{A}$ of Lemma 41 can be reformulated as follows: given a finite connected diagram $d:D\rightarrow\mathcal{A}$ and element $o\in\mathcal{O}_{n}(\lim d)$ , the following canonical morphism is an isomorphism

\[\overline{o}_{j}\rightarrow\int_{i:D}\overline{o\{p_{i}\}}_{j}\]

for any $j\in\{1,\dots,n\}$ .

Proof Note that, by definition, $\overline{o}_{j}=\alpha_{n,j}(\lim d,o)$ , and $\overline{o\{p_{i}\}}_{j}=\alpha_{n,j}(d_{i},o\{p_{i}\})$ . This is a direct application of Corollary 44.

Lemma 46 (Limits commute with dependent pairs). Given functors $K:I\rightarrow\mathrm{Set}$ and $G:\int K\rightarrow\mathrm{Set}$ , the following canonical morphism is an isomorphism

\[\coprod_{\alpha\in\lim K}\int_{i:I}G(i,p_{i}(\alpha))\rightarrow\int_{i:I}\coprod_{x\in Ki}G(i,x)\]

Proof The domain consists of a family $(\alpha_{i})_{i\in I}$ where $\alpha_{i}\in K_{i}$ together with a family $(g_{i})_{i\in I}$ where $g_{i}\in G(i,\alpha_{i})$ , such that that for each morphism $i\xrightarrow{u}j$ in I, we have $Ku(\alpha_{i})=\alpha_{j}$ and $(Gu)(g_{i})=g_{j}$ .

The codomain consists of a family $(x_{i},g_{i})_{i\in I}$ where $x_{i}\in Ki$ and $g_{i}\in G(i,x_{i})$ , such that for each morphism $i\xrightarrow{u}j$ in I, we have $Ku(x_{i})=x_{j}$ and $(Gu)(g_{i})=g_{j}$ .

The canonical morphism maps $((x_{i})_{i\in I},(g_{i})_{i\in I})$ to the family $(x_{i},g_{i})_{i\in I}$ . It is clearly a bijection.

Lemma 47. A coproduct $\coprod_{i}G_{i}$ of functors from a small category $\mathcal{B}$ with finite connected limits to Set preserves those limits if and only if each $G_{i}$ does.

Proof This is a consequence of the following statement, which is a direct application of Adámek et al. (Reference Adámek, Borceux, Lack and Rosicky2002, Theorem 2.4 and Example 2.3.(iii)): if $\mathscr{B}$ is a small category with finite connected limits, then a functor $G:\mathscr{B}\rightarrow\mathrm{Set}$ preserves those limits if and only if $\int G$ is a coproduct of filtered categories.

Proof of Lemma 41 Let $d:I\rightarrow\mathcal{A}$ be a finite connected diagram and X be a functor preserving finite connected limits. Then,

Thus, since X preserves finite connected limits by assumption,

(3.2) \begin{align}\int_{i}F(X)_{d_{i}} & =\coprod_{n}\coprod_{o\in\int_{i}\mathcal{O}_{n}(d_{i})}X_{\int_{i:I}\overline{p_{i}(o)}_{1}}\times\dots\times X_{\int_{i:I}\overline{p_{i}(o)}_{n}}\end{align}

Now, let us prove the only if statement first. Assume that each $\mathcal{O}_{n}$ and $\alpha_{n,i}:\int\mathcal{O}_{n}\rightarrow\mathcal{A}$ preserve finite connected limits. Then,

Conversely, let us assume that F restricts to an endofunctor on $\mathscr{C}$ . Then, $F(1)=\coprod_{n}\mathcal{O}_{n}$ preserves finite connected limits. By Lemma 47, each $\mathcal{O}_{n}$ preserves finite connected limits. By Corollary 45, it is enough to prove that given a finite connected diagram $d:D\rightarrow\mathcal{A}$ and element $o\in\mathcal{O}_{n}(\lim d)$ , the following canonical morphism is an isomorphism

\[\overline{o}_{j}\rightarrow\int_{i:D}\overline{o\{p_{i}\}}_{j}\]

Now, we have

On the other hand,

It follows from those two chains of isomorphisms that each function $X_{\overline{o}_{j}}\rightarrow X_{\int_{i:I}\overline{o\{p_{i}\}}_{j}}$ is a bijection, or equivalently (by the Yoneda Lemma), that $\mathscr{C}(K\overline{o}_{j},X)\rightarrow\mathscr{C}(K\int_{i:I}\overline{o\{p_{i}\}}_{j},X)$ is an isomorphism. Since the Yoneda embedding is fully faithful, $\overline{o}_{j}\rightarrow\int_{i:D}\overline{o\{p_{i}\}}_{j}$ is an isomorphism.

4.1 Rigid (rules and

The rules and handle non-cyclic unification of M(x) with $\boldsymbol{\Gamma};a\vdash o(\delta)$ for some $o\in\mathcal{O}_{n}(a)$ , where $M\notin\boldsymbol{\Gamma}$ . By Lemma 48, a unifier is given by a substitution $\sigma:\Gamma\rightarrow\Delta$ and a term u such that

(4.2) \begin{equation}o(\delta[\sigma])=u\{x\}.\end{equation}

Now, u is either some M(y) or $o'(\vec{v})$ . But in the first case, $u\{x\}=M(y)\{x\}=M(x\circ y)$ , contradicting Equation (4.2). Therefore, $u=o'(\delta')$ for some $o'\in\mathcal{O}_{n}(m)$ and $\delta'$ is a substitution from $\alpha_{o'}$ to $\Delta$ . Then, $u\{x\}=o'\{x\}(\delta\{x_{}^{o'}\})$ . It follows from Equation (4.2) that $o=o'\{x\}$ , and $\delta[\sigma]=\delta'\{x_{}^{o'}\}$ .

Note that there is at most one o’ such that $o=o'\{x\}$ , by Assumption 27.(i). In this case, a unifier is equivalently given by substitutions $\sigma:\Gamma\rightarrow\Delta$ and $\sigma':\alpha_{o'}\rightarrow\Delta$ such that $\delta[\sigma]=\sigma'\{x_{}^{o'}\}$ . But, by Lemma 48, this is precisely the data for a unifier of $\delta$ and $x^{o'}$ . This actually induces an isomorphism between the two categories of unifiers, thus justifying the rules and .

4.2 Flex (rule )

The rule handles unification of M(x) with N(y) where $M\neq N$ in a scope a. More explicitly, this is about computing the pushout of and $(X:a)\xrightarrow{\mathcal{L}y}(X:n)\xrightarrow{\cong}(N:n)$ .

Thanks to the following lemma, it is actually enough to compute the pushout of $\mathcal{L}x$ and $\mathcal{L}y$ , taking $A=(X:a)$ , $B=(X:m)$ , $C=(X:n)$ , $Y=\boldsymbol{\Gamma}\backslash M$ , so that $B+Y\cong\boldsymbol{\Gamma}$ , since coproduct is concatenation of metacontext by Corollary 18.

By Assumption 27.(ii), the pushout of $\mathcal{L}x$ and $\mathcal{L}y$ is the image by $\mathcal{L}$ of the pullback of x and y in $\mathcal{A}$ , thus justifying the rule .

5.1 Sequential unification (rules and )

The rule is a direct application of the following general lemma, since the empty metavariable context is initial: the only morphism to any metacontext $\Gamma$ is the empty metasubstitution ().

The rule is a direct application of a stepwise construction of coequalisers valid in any category, as noted by Rydeheard & Burstall (Reference Rydeheard and Burstall1988, Theorem 9): if the first two diagrams below are coequalisers, then the last one as well.

5.2 Flex-Flex, no cycle (rule )

The rule transitions from unification to pruning. While unification is a coequaliser construction, in Section 4, we explained that pruning is a pushout construction. The rule is justified by the following well-known connection between those two notions.

We take B to be $M:m$ and C to be $\boldsymbol{\Gamma}\backslash M$ . The premise $\boldsymbol{\Gamma}\backslash M\vdash_{}t\boldsymbol{:>}x\Rightarrow t';\sigma\dashv\Delta$ tells us that we have a pushout as below left. The conclusion $\boldsymbol{\Gamma}\vdash M(x)=t\Rightarrow M\mapsto t',\sigma\dashv\Delta$ states that we have a coequaliser as below right, in accordance with the above lemma.

5.3. Flex-Flex, same metavariable (rule )

Here, we detail unification of M(x) and M(y), for $x,y\in\hom_{\mathcal{A}}(m,a)$ . By Lemma 28, $M(x)=\mathcal{L}x[in_{M}]$ and $M(y)=\mathcal{L}y[in_{M}]$ . We exploit the following lemma with $u=\mathcal{L}x$ , $v=\mathcal{L}y$ , $D=\boldsymbol{\Gamma}\backslash M$ so that $B+D\cong\boldsymbol{\Gamma}$ since coproduct is concatenation of metacontext, by Corollary 18.

It follows that it is enough to compute the coequaliser of $\mathcal{L}x$ and $\mathcal{L}y$ . Furthermore, by Assumption 27.(ii), it is the image by $\mathcal{L}$ of the equaliser of x and y, thus justifying the rule .

5.4 Flex-rigid, cyclic (rule )

The rule handles unification of M(x) and a term t such that t is rigid and M occurs in t. In this section, we show that indeed there is no successful unifier. More precisely, we prove Corollary 59 below, stating that if there is a unifier of a term t and a metavariable application M(x), then either M occurs at top level in t, or it does not occur at all. The argument follows the basic intuition that $\sigma_{M}=t[M\mapsto\sigma_{M}]$ is impossible if M occurs deeply in u because the sizes of both hand sides can never match. To make this statement precise, we need some recursive definitions and properties of size.

We will also need to count the occurrences of a metavariables in a term.

6.1. Termination

In this section, we sketch an explicit argument to justify termination of our algorithm described in Figure 10. Note that the pruning and the unification phases are not mutually recursive: the latter depends on the former, but not conversely. Therefore, we can first show that the pruning phase terminates, and then that the unification does. Both phases involve three recursive calls (cf. the rules , , , and ). In each phase, the second recursive call for splitting is not structurally recursive, making Agda unable to check termination. However, we can devise an adequate notion of input size so that for each recursive call, the inputs are strictly smaller than the inputs of the calling site. First, we define the size $|\boldsymbol{\Gamma}|$ of a proper metacontext $\boldsymbol{\Gamma}$ as its length, while $|\bot|=0$ by definition. We also recursively define the sizeFootnote ⁷ $||t||$ of a proper term t by $||M(x)||=1$ and $||o(\vec{t})||=1+||\vec{t}||$ , with $||\vec{t}||=\sum_{i}||t_{i}||$ . We also define the size of the error term $||\mbox{!}||$ as 1. Note that no term is of size 0.

Let us first quickly justify termination of the pruning phase.

Proof Consider the above defined size of the input, which is a term t for , or a list of terms $\vec{t}$ for . It is straightforward to check that the sizes of the inputs of recursive calls are strictly smaller thanks to the previous lemma. Let us detail the case of the rule . We show that the input $\delta[\sigma_{1}]$ of the second recursive call is smaller than the original input $t,\delta$ . Then,

For termination of the main unification phase, we consider the size of the input to be the (lexicographic) pair $(|\Gamma|,||t||)$ for or $(|\Gamma|,||\vec{t}||)$ for , given as input a pair of terms (t,t’) or lists of terms $(\vec{t},\vec{t'})$ in the metacontext $\Gamma$ .

If $|\Delta|<|\Gamma|$ , then we are done. Otherwise, by Lemma 64, we have $|\Delta|=|\Gamma|$ , and then,

6.2. Completeness

In this section, we explain why soundness (Sections 4 and 5) and termination (Section 6.1) entail completeness. Intuitively, one may worry that the algorithm fails in cases where it should not. In fact, we already checked in the previous sections that failure only occurs when there is no unifier, as expected. Indeed, failure is treated as a free “terminal” unifier, as explained in Section 3.1, by considering the category $\mathrm{MCon}(S)_{\bot}$ extending category $\mathrm{MCon}(S)$ with an error metacontext $\bot$ . Corollary 19 implies that since the algorithm terminates and computes the coequaliser in $\mathrm{MCon}(S)_{\bot}$ , it always finds the most general unifier in $\mathrm{MCon}(S)$ if it exists, and otherwise returns failure (i.e., the map to the terminal object $\bot$ ).

7.1 Metavariable arguments as sets

If we think of the arguments of a metavariable as specifying the available variables in any order (as in Section 2.1), then it makes sense to assemble them in a set rather than in a list. This motivates considering the category $\mathcal{A}=\mathbb{I}$ whose objects are natural numbers and a morphism $n\rightarrow p$ is a subset of $\{1,\dots,p\}$ of cardinality n. Equivalently, $\mathbb{I}$ can be taken as subcategory of $\mathbb{F}_{m}$ consisting of strictly increasing injections, or as the subcategory of the augmented simplex category consisting of injective functions. Then, a metavariable takes as argument a set of variables, rather than a list of distinct variables. In this approach, unifying two metavariables (see the rules and ) amount to computing a set intersection.

7.2 Simply typed $\lambda$ -calculus

In this section, we present the example of simply typed $\lambda$ -calculus. Our treatment generalises to any multi-sorted binding signature (Fiore & Hur Reference Fiore and Hur2010).

Let T denote the set of simple types generated by a set of base types and a binary arrow type construction $-\Rightarrow-$ . Let us now describe the category $\mathcal{A}$ of arities, or scopes, and renamings between them. An arity $\vec{\sigma}\rightarrow\tau$ consists of a list of input types $\vec{\sigma}$ and an output type $\tau$ . A term t in $\vec{\sigma}\rightarrow\tau$ considered as a scope is intuitively a well-typed term t of type $\tau$ potentially using variables whose types are specified by $\vec{\sigma}$ . A valid choice of arguments for a metavariable $M:(\vec{\sigma}\rightarrow\tau)$ in scope $\vec{\sigma}'\rightarrow\tau'$ first requires $\tau=\tau'$ and consists of an injective renaming $\vec{r}$ between $\vec{\sigma}=(\sigma_{1},\dots,\sigma_{m})$ and $\vec{\sigma}'=(\sigma'_{1},\dots,\sigma'_{n})$ , that is, a choice of distinct positions $(r_{1},\dots,r_{m})$ in $\{1,\dots,n\}$ such that $\vec{\sigma}=\sigma'_{\vec{r}}$ .

This discussion determines the category of arities as $\mathcal{A}=\mathbb{F}_{m}[T]\times T$ , where $\mathbb{F}_{m}[T]$ is the category of finite lists of elements of T and injective renamings between them. Table 1 summarises the GB-signature specifying the syntax, where $|\vec{\sigma}|_{\tau}$ denotes the number (as a cardinal set) of occurrences of $\tau$ in $\vec{\sigma}$ . The middle column associates a subset of the operation symbols to each typing rule: the total set of operation symbols is recovered by computing the (disjoint) union of them. The last column gives the list $\alpha_{o}=(\overline{o}_{1},\dots,\overline{o}_{n})$ for each operation symbol o.

Table 1. Simply typed $\lambda$ -calculus (Section 7.2)

Next, every $\mathcal{O}_{n}$ preserve finite connected limits by Lemma 47 because they are all coproduct of representable functors (as shown below), which preserve limits by Mac Lane (Reference Mac Lane1998, Theorem V.4.1):

\begin{align*}\mathcal{O}_{0}(-) & \cong\coprod_{\tau\in T}\hom_{\mathcal{A}}(\tau\rightarrow\tau,-)\\\mathcal{O}_{1}(-) & \cong\coprod_{\tau_{1},\tau_{2}\in T}\hom_{\mathcal{A}}(\cdot\rightarrow\tau_{1}\Rightarrow\tau_{2},-)\\\mathcal{O}_{2}(-) & \cong\coprod_{\tau,\tau'\in T}\hom_{\mathcal{A}}(\cdot\rightarrow\tau,-)\\\mathcal{O}_{3+n}(-) & \cong\emptyset\end{align*}

Note that $\alpha_{3+n,i}:\emptyset\rightarrow\mathcal{A}$ vacuously preserves finite connected limits. It remains to show that $\alpha_{1,1}$ , $\alpha_{2,1}$ , and $\alpha_{2,2}$ also preserve them. First, we observe that the domains of those functors are isomorphic to $\mathbb{F}_{m}[T]\times T^{2}$ . More specifically,

\begin{align*}\int\mathcal{O}_{i} & \cong\mathbb{F}_{m}[T]\times T^{2} & i\in\{1,2\}\\(\vec{\sigma}\rightarrow\tau_{1}\Rightarrow\tau_{2},l_{\tau_{1},\tau_{2}}) & \mapsto(\vec{\sigma},\tau_{1},\tau_{2}) & i=1\\(\vec{\sigma}\rightarrow\tau,a_{\tau'}) & \mapsto(\vec{\sigma},\tau,\tau') & i=2\end{align*}

Precomposed with the reverse isomorphisms, the functors $\alpha_{1,1},\alpha_{2,1},\alpha_{2,2}:\int\mathcal{O}_{n}\rightarrow\mathcal{A}$ induce the three below left functors $\beta_{1,1}$ , $\beta_{2,1}$ , and $\beta_{2,2}$ .

Now, finite connected limits in $\mathbb{F}_{m}[T]\times T^{n}$ are computed in $\mathbb{F}_{m}[T]$ . Therefore, to conclude that the functors $\beta_{n,i}$ listed above left preserve those limits, it is enough to show that given $\tau_{1},\tau_{2}\in T$ , the compositions $\mathbb{F}_{m}[T]\xrightarrow{(-,\tau_{1},\tau_{2})}\mathbb{F}_{m}[T]\times T^{2}\xrightarrow{\beta_{n,i}}\mathbb{F}_{m}[T]\times T\rightarrow\mathbb{F}_{m}[T]$ listed above right do so. It is obvious for the last two functors, and it is straightforward to check for the first functor, by investigating the constructions of equalisers and pullbacks.

It follows that the generic pattern unification algorithm applies. Equalisers and pullbacks are computed following the same pattern as in pure $\lambda$ -calculus. For example, to unify $M(\vec{x})$ and $M(\vec{y})$ , we first compute the vector $\vec{z}$ of common positions between $\vec{x}$ and $\vec{y}$ , thus satisfying $x_{\vec{z}}=y_{\vec{z}}$ . Then, the most general unifier maps $M:(\vec{\sigma}\rightarrow\tau)$ to the term $P(\vec{z})$ , where the arity $\vec{\sigma}'\rightarrow\tau'$ of the fresh metavariable P is the only possible choice such that $P(\vec{z})$ is a valid term in the scope $\vec{\sigma}\rightarrow\tau$ , that is, $\tau'=\tau$ and $\vec{\sigma}'=\sigma_{\vec{z}}$ .

7.3 Simply typed $\lambda$ -calculus modulo $\beta$ $\eta$

In this section, we explain how we account for Miller’s original setting: simply typed $\lambda$ -calculus modulo $\beta$ and $\eta$ -equations. We follow Cheney’s presentation of the equation-free syntax of $\beta$ -short $\eta$ -long normal forms with metavariables (Cheney, Reference Cheney2005, Section 2.1). The normals forms have the following shape, where z denotes a variable, $\vec{x}$ denotes a vector of distinct variables and M denotes a metavariable:

\[t::=\lambda\vec{y}.z\vec{t}|\lambda\vec{y}.M\vec{x}\]

This syntax is closed under substitution of metavariables, modulo $\beta$ -normalisation. For example, replacing M by $\lambda ww'.z\vec{t}$ in $\lambda\vec{y}.Mxx'$ yields a $\beta$ -reducible term $\lambda\vec{y}.(\lambda ww'.z\vec{t})xx'$ which yields a normal form after two $\beta$ -reductions.

We now explain how this syntax of normal forms can be specified by a GB-signature. The associated metavariable substitution is then hereditary in the sense of Remark 68, since the syntax generated by any GB-signature is always stable under metavariable substitution.

Notation 69. We denote a type $\sigma_{1}\Rightarrow\dots\Rightarrow\sigma_{n}\Rightarrow\iota$ by $\vec{\sigma}\Rightarrow\iota$ , where $\iota$ is a base type. Note that any type can be written in this way, uniquely.

We take the same notion of scope as in the previous section: a term well-formed in scope $\vec{\sigma}\rightarrow\tau$ will correspond to a normal form of type $\tau$ with free variables of types $\vec{\sigma}$ .

We choose a notion of scope morphism different from that of the previous section, so that the introduction rule of metavariables matches the below typing rule for a metavariable term $\lambda\vec{y}.M\vec{x}$ , given a metavariable M of type $\vec{\tau}\Rightarrow\iota$ .

Assuming that the arity of M is $\vec{\tau}_{1}\rightarrow\vec{\tau_{2}}\Rightarrow\iota$ with $\vec{\tau}=\vec{\tau}_{1},\vec{\tau}_{2}$ , this means that a scope morphism from $\vec{\tau}_{1}\rightarrow(\vec{\tau_{2}}\Rightarrow\iota)$ to $\vec{\sigma}\rightarrow(\vec{\sigma'}\Rightarrow\iota')$ should be empty unless $\iota=\iota'$ . In that case, a morphism is given by a choice of distinct variables of types $\vec{\tau}=\vec{\tau_{1}},\vec{\tau}_{2}$ in $\vec{\sigma},\vec{\sigma}'$ , that is, a morphism in $\mathbb{F}_{m}[T]$ between those two lists of types. We therefore arrive to the following definition.

This definition readily entails that $\mathcal{A}$ is equivalent to $\mathbb{F}_{m}[T]\times B$ .

We have specified the category of scopes so that we get a suitable introduction rule for metavariables. It remains to provide a GB-signature on top of $\mathcal{A}$ that accounts for the construction $\lambda\vec{y}.x\vec{t}$ , with the corresponding below typing rule:

\[\dfrac{x:(\vec{\tau}\Rightarrow\iota)\in(\vec{z}:\vec{\sigma},\vec{y}:\vec{\sigma'})\qquad\vec{z}:\vec{\sigma},\vec{y}:\vec{\sigma'}\vdash\vec{t}:\vec{\tau}}{\vec{z}:\vec{\sigma}\vdash\lambda\vec{y}.x\vec{t}:\vec{\sigma'}\Rightarrow\iota}\]

The following components induce a GB-signature that generates the same base syntax:

\begin{align*}O(\vec{\sigma}\rightarrow\tau) & =\{l_{x,\tau_{1},\dots,\tau_{n},\vec{\sigma'},\iota}|\tau=(\vec{\sigma'}\Rightarrow\iota),x\in|\vec{\sigma},\vec{\sigma'}|_{\vec{\tau}\Rightarrow\iota}\}\\\alpha_{l_{x,\tau_{1},\dots,\tau_{n},\vec{\sigma'},\iota}} & =\left(\begin{array}{c}\vec{\sigma},\vec{\sigma'}\rightarrow\tau_{1}\\\dots\\\vec{\sigma},\vec{\sigma'}\rightarrow\tau_{n}\end{array}\right)\end{align*}

7.4 Ordered $\lambda$ -calculus

Our setting handles linear ordered $\lambda$ -calculus, consisting of $\lambda$ -terms using all the variables in context. In this context, a metavariable M of arity $m\in\mathbb{N}$ can only be used in the scope m, and there is no freedom in choosing the arguments of a metavariable application, since all the variables must be used, in order. Thus, there is no need to even mention those arguments in the syntax. It is thus not surprising that ordered $\lambda$ -calculus is already handled by first-order unification, where metavariables do not take any argument, by considering ordered $\lambda$ -calculus as a multi-sorted Lawvere theory where the sorts are the scopes, and the syntax is generated by operations $L_{n}\times L_{m}\rightarrow L_{n+m}$ and abstractions $L_{n+1}\rightarrow L_{n}$ .

Our generalisation can handle calculi combining ordered and unrestricted variables, such as the calculus underlying ordered linear logic described in Polakow and Pfenning (Reference Polakow and Pfenning2000). In this section, we detail this specific example. Note that this does not fit into Schack–Nielsen and Schürman’s pattern unification algorithm (Schack-Nielsen &Schürmann, Reference Schack-Nielsen and Schürmann2010) for linear types where exchange is allowed (the order of their variables does not matter).

The set T of types is generated by a set of atomic types and two binary arrow type constructions $\Rightarrow$ and $\twoheadrightarrow$ . The syntax extends pure $\lambda$ -calculus with a distinct application $t^{>}\ u$ and abstraction $\lambda^{>}u$ . Variables contexts are of the shape $\vec{\sigma}|\vec{\omega}\rightarrow\tau$ , where $\vec{\sigma}$ , $\vec{\omega}$ , and $\tau$ are taken in T. The idea is that a term in such a context has type $\tau$ and must use all the variables of $\vec{\omega}$ in order but is free to use any of the variables in $\vec{\sigma}$ . Assuming a metavariable M of arity $\vec{\sigma}|\vec{\omega}\rightarrow\tau$ , the above discussion about ordered $\lambda$ -calculus justifies that there is no need to specify the arguments for $\vec{\omega}$ when applying M. Thus, a metavariable application $M(\vec{x})$ in the scope $\vec{\sigma}'|\vec{\omega}'\rightarrow\tau'$ is well-formed if $\tau=\tau'$ and $\vec{x}$ is an injective renaming from $\vec{\sigma}$ to $\vec{\sigma}'$ . Therefore, we take $\mathcal{A}=\mathbb{F}_{m}[T]\times T^{*}\times T$ for the category of arities, where $\mathbb{F}_{m}[T]$ accounts for the unrestricted variables, while $T^{*}$ accounts for the linear ones: it denotes the discrete category whose objects are lists of elements of T. The remaining components of the GB-signature are specified in Table 2, following the same convention as in Table 1. We alternate typing rules for the unrestricted and the ordered fragments (variables, application, abstraction).

Table 2. Ordered $\lambda$ -calculus (Section 7.4)

Pullbacks and equalisers are computed essentially as in Section 7.2. For example, the most general unifier of $M(\vec{x})$ and $M(\vec{y})$ maps M to $P(\vec{z})$ where $\vec{z}$ is the vector of common positions of $\vec{x}$ and $\vec{y}$ , and P is a fresh metavariable of arity $\sigma_{\vec{z}}|\vec{\omega}\rightarrow\tau$ .

7.5 Intrinsic polymorphic syntax

7.5.1 Syntactic System F

We present intrinsic System F, in the spirit of Hamana (Reference Hamana2011). The Agda implementation of the pattern-friendly GB-signature can be found in the Supplementary Material.

The syntax of types in type scope n is inductively generated as follows, following the de Bruijn level convention:

\[\dfrac{1\leq i\leq n}{n\vdash i}\qquad\dfrac{n\vdash t\quad n\vdash u}{n\vdash t\Rightarrow u}\qquad\dfrac{n+1\vdash t}{n\vdash\forall t}\]

Let $S:\mathbb{F}_{m}\rightarrow\mathrm{Set}$ be the functor mapping n to the set $S_{n}$ of types for system F taking free type variables in $\{1,\dots,n\}$ . In other words, $S_{n}=\{\tau|n\vdash\tau\}$ . Intuitively, a metavariable arity $n|\vec{\sigma}\rightarrow\tau$ specifies the number n of free type variables, the list of input types $\vec{\sigma}$ , and the output type $\tau$ , all living in $S_{n}$ . This provides the underlying set of objects of the category $\mathcal{A}$ of arities. A term t in $n|\vec{\sigma}\rightarrow\tau$ considered as a scope is intuitively a well-typed term of type $\tau$ potentially involving ground variables of type $\vec{\sigma}$ and type variables in $\{1,\dots,n\}$ .

A metavariable $M:(n|\sigma_{1},\dots,\sigma_{p}\rightarrow\tau)$ in the scope $n'|\vec{\sigma}'\rightarrow\tau'$ must be supplied with a choice $(\eta_{1},\dots,\eta_{n})$ of n distinct type variables among the set $\{1,\dots n'\}$ such that $\tau[\vec{\eta}]=\tau'$ , as well as an injective renaming $\vec{\sigma}[\vec{\eta}]\rightarrow\vec{\sigma}'$ , that is, a list of distinct positions $r_{1},\dots,r_{p}$ such that $\vec{\sigma}[\vec{\eta}]=\sigma'_{\vec{r}}$ .

This defines the data for a morphism in $\mathcal{A}$ between $(n|\vec{\sigma}\rightarrow\tau)$ and $(n'|\vec{\sigma}'\rightarrow\tau')$ . The intrinsic syntax of system F can then be specified as in Table 3, following the same convention as in Table 1. The induced GB-signature is pattern-friendly. For example, morphisms in $\mathcal{A}$ are easily seen to be monomorphic; we detail in Section 7.5.3 the proof that $\mathcal{A}$ has finite connected limits.

Table 3. The (pattern-friendly) GB-signature of (syntactic) System F (Section 7.5.1)

Pullbacks and equalisers in $\mathcal{A}$ are essentially computed as in Section 7.2, by computing the vector of common (value) positions. For example, given a metavariable M of arity $m|\vec{\sigma}\rightarrow\tau$ , to unify $M(\vec{w}|\vec{x})$ with $M(\vec{y}|\vec{z})$ , we compute the vector of common positions $\vec{p}$ between $\vec{w}$ and $\vec{y}$ , and the vector of common positions $\vec{q}$ between $\vec{x}$ and $\vec{z}$ . Then, the most general unifier maps M to the term $P(\vec{p}|\vec{q})$ , where P is a fresh metavariable. Its arity $m'|\vec{\sigma}'\rightarrow\tau'$ is the only possible one for $P(\vec{p}|\vec{q})$ to be well-formed in the scope $m|\vec{\sigma}\rightarrow\tau$ , that is, m’ is the size of $\vec{p}$ , while $\tau'=\tau[p_{i}\mapsto i]$ and .

7.5.2 System F modulo $\beta\eta$

In this section, we sketch how we can handle System F modulo $\beta\eta$ in the spirit of Section 7.3, by devising a signature for normal forms. To make the syntax more legible, we depart from the previous presentation and instead consider System F as a pure type system. We also ignore the de Bruijn encoding. A scope is now of the shape $\vec{y}:\vec{u}\rightarrow\tau$ , where

• • $\vec{y}:\vec{u}$ is a list of variable declarations $y_{1}:u_{1},\dots,y_{n}:u_{n}$ where $u_{i}$ is either $*$ , meaning that $y_{i}$ is a type variable, or a type which is well-formed in context involving all the type variables occuring before $y_{i}$ in the scope;

• • • $\tau$ is a type well-formed in $\vec{y}:\vec{u}$ .

We use the notation $\prod(\alpha:u).\tau$ , where $\tau$ may depend on $\alpha$ , to mean either $\forall\alpha.\tau_{2}$ in case $u=*$ , or $u\Rightarrow\tau$ otherwise (in the latter case, $\tau$ does not depend on $\alpha$ ).

Note that any type can be written as $\prod(y_{1}:u_{1})\prod\dots\prod(y_{n}:u_{n}).\iota$ , abbreviated as $\prod(\vec{y}:\vec{u}).\iota$ , where $\iota$ is a type variable. Any scope $(\vec{y}:\vec{u})\rightarrow\prod(\vec{z}:\vec{v}).\iota$ , induces a type $\prod(\vec{y}:\vec{u})(\vec{z}:\vec{v}).\iota$ . A morphism between two scopes inducing the types $\prod(\vec{y}:\vec{u}).\iota$ and $\prod(\vec{z}:\vec{v}).\iota'$ is an injective renaming $\rho$ between $\vec{y}:\vec{u}$ and $\vec{z}:\vec{v}$ such that $\iota[\rho]=\iota'$ .

Let us now describe the base syntax. We write $\Gamma\vdash t:*$ to mean that t is a type well-formed in $\Gamma$ . We do not make any syntactic distinction between type and term abstractions.

The base syntax is generated by the following rule, where $\iota$ denotes a type variable, and $u_{i}$ or $v_{i}$ are either types or $*$ .

\[\dfrac{\Gamma,\vec{y}:\vec{u}\vdash x:\iota}{\Gamma\vdash\lambda\vec{y}.x:\prod(\vec{y}:\vec{u}).\iota}\qquad\dfrac{\begin{array}{cc} & \Gamma,\vec{y}:\vec{u}\vdash x:\prod(\alpha_{1}:v_{1}).\tau_{1}\\\Gamma,\vec{y}:\vec{u}\vdash t_{1}:v_{1} & \tau_{1}[\alpha_{1}\mapsto t_{1}]=\prod(\alpha_{2}:v_{2}).\tau_{2}\\\Gamma,\vec{y}:\vec{u}\vdash t_{2}:v_{2} & \tau_{2}[\alpha_{2}\mapsto t_{2}]=\prod(\alpha_{3}:v_{3}).\tau_{3}\\\dots & \tau_{n}[\alpha_{n}\mapsto t_{n}]=\iota\end{array}}{\Gamma\vdash\lambda\vec{y}.x\vec{t}:\prod(\vec{y}:\vec{u}).\iota}\]

Let us now describe the enriched syntax. We write $M::\prod(\vec{y}:\vec{u}).\iota$ to mean that the type induced by the arity of M is $\prod(\vec{y}:\vec{u}).\iota$ . The introduction rule for metavariables is the following:

As in Section 7.3, thanks to our modified notion of scope morphism, this rule indeed complies with our introduction rule for metavariables, in the sense that it requires the same data.