Denotational semantics as a foundation for cost recurrence extraction for functional languages

A standard informal method for analyzing the asymptotic complexity of a program is to extract a recurrence that describes its cost in terms of the size of its input, and then to compute a closed-form upper bound on that recurrence. We give a formal account of that method for functional programs in a higher-order language with let-polymorphism. The method consists of two phases. In the first phase, a monadic translation is performed to extract a cost-annotated version of the original program. In the second phase, the extracted program is interpreted in a model. The key feature of this second phase is that different models describe different notions of size. This plays out in several ways. For example, when analyzing functions that take arguments of inductive types, different notions of size may be appropriate depending on the analysis. When analyzing polymorphic functions, our approach shows that one can formally describe the notion of size of an argument in terms of the data that is common to the notions of size for each type instance of the domain type. We give several examples of different models that formally justify various informal cost analyses to show the applicability of our approach.


Introduction
The method for analyzing the asymptotic cost of a (functional) program f (x) that is typically taught to introductory undergraduate students is to extract a recurrence T f (n) that describes an upper bound on the cost of f (x) in terms of the size of x, and then establish a non-recursive upper bound on T f (n) (we will focus on upper bounds, but much of what we say holds mutatis mutandis for lower bounds, and hence tight bounds). The goal of this work is to put the process of this informal approach to cost analysis on firm mathematical footing. Of course, various formalizations of cost analysis have been discussed for almost as long as there has been a distinct subfield of Programming Languages. Most of the recent work in this area is focused on developing formal techniques for cost analysis that enable the (possibly automated) analysis of as large a swath of programs as possible. In doing so, the type systems and the logics used grow ever more complex. There is work that incorporates size and cost into type information, for example by employing refinement types or type-and-effect systems. There is work that formalizes reasoning about cost in program logics such as separation logic with time credits. But as witnessed by most undergraduate texts on algorithm analysis, complex type systems and separation logic are not commonly taught. Instead, a function of some form (a recurrence) that computes the cost in terms of the size of the argument is extracted from the source code. This is the case for "simple" compositional worst-case analyses, but also more more complex techniques. For example, the banker's and physicist's methods of amortized analysis likewise proceed by extracting a function to describe cost; the notion of cost itself, and the extraction of a suitably precise cost function, is more complex, but broadly speaking the structure of the analysis is the same. That is the space we are investigating here: how do we justify that informal process? 1 The justification might not itself play a role in applying the technique informally, any more than we require introductory students to understand the theory behind a type inference algorithm in order to informally understand why their programs typecheck. But certainly that theory should be settled. Our approach is through denotational semantics, which, in addition to justifying the informal process, also helps to explicate a few questions, such as why length is an appropriate measure of size for cost recurrences for polymorphic list functions (a question that is close to, but not quite the same as, parametricity).
Turning to the technical development, in previous work [Danner et al., 2013[Danner et al., , 2015 we have developed a recurrence extraction technique for higher-order functional programs for which the bounding is provable that is based on work by Danner and Royer [2007]. The technique is described as follows: (1) We define what is essentially a monadic translation into the writer monad from a callby-value source language that supports inductive types and structural recursion (fold) to a call-by-name recurrence language; we refer to programs in the latter language as syntactic recurrences. The recurrence language is axiomatized by a size (pre)order rather than equations. The syntactic recurrence extracted from a source-language program f (x) describes both the cost and result of f (x) in terms of x.
(2) We define a bounding relation between source language programs and syntactic recurrences. The bounding relation is a logical relation that captures the notion that the syntactic recurrence is in fact a bound on the operational cost and the result of the source language program. This notion extends reasonably to higher-type, where highertype arguments of a syntactic recurrence are thought of recurrences that are bounds on the corresponding arguments of the source language program. We then prove a bounding theorem that asserts that every typeable program in the source language is related to the recurrence extracted from it. (3) The syntactic recurrence is interpreted in a model of the recurrence language. This is where values are abstracted to some notion of size; e.g., the interpretation may be defined so that a value of inductive type δ is interpreted by the number of δ-constructors in v. We call the interpretation of a syntactic recurrence a semantic recurrence, and it is the semantic recurrences that are intended to match the recurrences that arise from informal analyses.
In this paper, we extend the above approach in several ways. First and foremost, we investigate the models, semantic recurrences, and size abstraction more thoroughly than in previous work, and show how different models can be used to formally justify typical informal extract-and-solve cost analyses. Second, we add ML-style let-polymorphism, and adapt the techniques to an environment-based operational semantics, a more realistic foundation for implementation than the substitution-based semantics used in previous work. In recent work, we have extended the technique for source languages with call-by-name and general recursion [Kavvos et al., 2020], and for amortized analyses [Cutler et al., 2020]; we do not consider these extensions in the main body of this paper, in order to focus on the above issues in isolation.
Our source language, which we describe in Section 2, is a call-by-value higher-order functional language with inductive datatypes and structural recursion (fold) and ML-style let-polymorphism. That is, let-bound identifiers may be assigned a quantified type, provided that type is instantiated at quantifier-free types in the body of the let expression. Restricting to polymorphism that is predicative (quantifiers can be instantiated only with non-quantified types), first-order (quantifiers range over types, not type constructors), and second-class (polymorphic functions cannot themselves be the input to other functions) is sufficient to program a number of example programs, without complicating the denotational models used to analyze them. We define an environment-based operational semantics where each rule is annotated with a cost. For simplicity, we only "charge" for each unfolding of a recursive call, but the technique extends easily for any notion of cost that can be defined in terms of evaluation rules. We could also replace the rule-based cost annotations with a "tick" construct that the programmer inserts at the code points for which a charge should be made, though this requires the programmer to justify the cost model.
The recurrence language in Section 3 is a call-by-name λ-calculus with explicit predicative polymorphism (via type abstraction and type application) and an additional type for costs. Ultimately we care only about the meaning of a syntactic recurrence, not so much any particular strategy for evaluating it, and such a focus on mathematical reasoning makes call-by-name an appropriate formalism. The choice of explicit predicative polymorphism instead of let-polymorphism is minor, but arises from the same concerns: our main interest is in the models of the language, and it is simpler to describe models of the former than of the latter. To describe the recurrence language as call-by-name is not quite right, because the verification of the bounding theorem that relates source programs to syntactic recurrences does not require an operational semantics. Instead it suffices to axiomatize the recurrence language by a preorder, which we call the size order. The size order is defined in Figure 11, and a brief glimpse will show the reader that the axioms primarily consist of directed versions of the standard call-by-name equations. This is the minimal set of axioms necessary to verify the bounding theorem, but as we discuss more fully when we investigate models of the recurrence language, there is more to the size order than that. In a nutshell, a model in which the size order axiom for a given type constructor is non-trivial (i.e., in which the two sides are not actually equal) is a model that genuinely abstracts that particular type constructor to a size.
We can think of the cost type in the recurrence language as the "output" of the writer monad, and the recurrence extraction function that we give in Section 4 as the call-by-value monadic translation of the source language. In some sense, then, the recurrence extracted from a source program is just a cost-annotated version of the program. However, we think of the "program" part of the syntactic recurrence differently: it represents the size of the source program. Thus the syntactic recurrence simultaneously describes both the cost and size of the original program, what we refer to as a complexity. It is no surprise that we must extract both simultaneously, if for no other reason than compositionality, because if we are to describe cost in terms of size, then the cost of f (g(x)) depends on both the cost and size of g(x). Thinking of recurrence extraction as a call-by-value monadic translation gives us insight into how to think of the size of a function: it is a mapping from sizes (of inputs) to complexities (of computing the result on an input of that size). This leads us to view size as a form of usage, or potential cost, and it is this last term that we adopt instead of size. 2 The bounding relation e E that we define in Section 5 is a logical relation between source programs e and syntactic recurrences E. A syntactic recurrence is really a complexity, and e E says that the operational cost of e is bounded by the cost component of E and that the value of e is bounded by the potential component of E. The Bounding Theorem (Theorem 2.1) tells us that every typeable program is bounded by the recurrence extracted from it. Its proof is somewhat long and technical, but follows the usual pattern for verifying the Fundamental Theorem for a logical relation, and the details are in Appendix C.
In the recurrence language, the "data" necessary to describe the size has as much information as the original program; in the semantics we can abstract away as much or as little of this information as necessary. After defining environment models of the recurrence language in Section 6 (following Bruce et al. [1990]), in Section 7 we give several examples to demonstrate that different size abstractions result in semantic recurrences that formally justify typical extract-and-solve analyses. We stress that we are not attempting to analyze the cost of heretofore unanalyzed programs. Our goal is a formal process that mirrors as closely as possible the informal process we use at the board and on paper. The main examples demonstrate analyses where: (1) The size of a value v of inductive type δ is defined in terms of the number of δconstructors in v. For example, a list is measured by its length, a tree by either its size or its height, etc., enabling typical size-based recurrences (Section 7.2).
(2) The size of a value v of inductive type δ is (more-or-less) the number of constructors of every inductive type in v. For example, the size of a nat tree t is the number of nat tree constructors in t (its usual "size") along with the maximum number of nat constructors in any node of t, enabling the analysis of functions with more complex costs, such as the function that sums the nodes of a nat tree (Section 7.3). (3) A polymorphic function can be analyzed in terms of a notion of size that is more abstract than that given by its instances (Section 7.4). For example, while the size of a nat tree may be a pair (k, n), where k is the maximum key value and n the size (e.g., to permit analysis of the function that sums all the nodes), we may want the domain of the recurrence extracted from a function of type α tree → ρ to be N, corresponding to counting only the tree constructors. (4) We make use of the fact that the interpretation of the size order just has to satisfy certain axioms to derive recurrences for lower bounds. As an example, we parlay this into a formal justification for the informal argument that map(f • g) is more efficient than (map f ) • (map g) (Section 7.5). These examples end up clarifying the role of the size order, as mentioned earlier. It is not just the rules necessary to drive the proof of the syntactic bounding theorem, but a non-trivial interpretation of ≤ σ (i.e., one in which e ≤ σ e ′ is valid but not e = e ′ ) tells us that we have a model with a non-trivial size abstraction for σ. This clarification highlights interesting analogies with abstract interpretation: (1) when a datatype δ = µt.F is interpreted by a non-trivial size abstraction, there is an abstract interpretation between δ (abstract) and F [δ] (concrete); (2) interpreting the recurrence extracted from a polymorphic function in terms of a more abstract notion of size is possible if there is an abstract interpretation between two models.
The remaining sections of the paper discuss recent work in cost analysis and how our work relates to it, as well as limitations of and future directions for our approach.

The source language
The source language that serves as the object language of our recurrence extraction technique is a higher-order language with inductive types, structural iteration (fold) over those types, and ML-style polymorphism (i.e., predicative polymorphism, with polymorphic identifiers introduced only in let bindings), with an environment-based operational semantics that approximates typical implementation. This generalizes the source language of Danner et al. [2015], which introduced the technique for a monomorphically-typed language with a substitution-based semantics. We address general recursion in Section 8.
The grammar and typing rules for expressions are given in Figure 1. Type assignment derives (quantifier-free) types for expressions given a type context that assigns type schemes (quantified types) to identifiers. We write for the empty type context. Values are not a subset of expressions because, as one would expect in implementation, a function value consists of a function expression along with a value environment: a binding of free variables to values. The same holds for values of suspension type, and we refer to any pair of an expression and a value environment for its free variables as a closure (thus we use closure more freely than the usual parlance, in which it is restricted to functions). We adopt the notation common in the explicit substitution literature (e.g., Abadi et al. [1991]) and write vθ for a closure with value environment θ. Since the typing for map and mapv expressions depend on values, this requires a separate notion of typing for values, which in turn depends on a notion of typing for closures. These are defined in Figures 2 and 3. There is nothing deep in the typing of a closure value vθ under context Γ. Morally the rules just formalize that v can be assigned the expected type without regard to θ and that θ(x) is of type Γ(x). But since type contexts may assign type schemes, whereas type assignment only derives types, the formal definition is that θ(x) can be assigned any instance of Γ(x). We will freely assume notation for n-ary sums and products and their corresponding introduction and elimination forms, such as σ 0 ×σ 1 ×σ 2 , (e 0 , e 1 , e 2 ), and π 1 e for (σ 0 × σ 1 )× σ 2 , ((e 0 , e 1 ), e 2 ), and π 1 (π 0 e), respectively. We write fv(e) for the free variables of e and ftv(τ ) for the free type variables of τ .
Inductive types are defined by shape functors, ranged over by the metavariable F ; a generic inductive type has the form µt.F . If F is a shape functor and σ a type, then F [σ] is the result of substituting σ for free occurrences of t in F (the µ operator binds t, of course). Formally a shape functor is just a type, and so when certain concepts are defined by induction on type, they are automatically defined for shape functors as well. In the syntax for shape functors, t is a fixed type variable, and hence simultaneous nested definitions are not allowed. That is, types such as µt.unit + µs.(unit + s × t) are forbidden. However, an inductive type can be used inside of other types via the constant functor (σ) production of F , e.g. coding the type (α list) list as µt.unit + (µt.unit + α × t) × t. This restriction is just to simplify the presentation of the languages and models, and lifting it does not require fundamental changes. Figure 4 gives a number of types and values that we will use ρ, σ ∈ Type : in examples. We warn the reader that because most models of the recurrence language have non-standard interpretations of inductive types, the types σ and µt.σ may be treated very differently even when t is not free in σ. Thus it can actually make a real difference whether we define bool to be unit + unit or µt.unit + unit (if every type that would be defined by an ML datatype declaration were implemented as a possibly degenerate inductive type).
For every inductive type δ = µt.F there is an associated constructor c δ , destructor d δ , and iterator fold δ . Thought of informally as term constants, the first two have the typical types F [δ] → δ and δ → F [δ], but the type of fold δ is somewhat non-standard: δ → (F [σ susp] → σ) → σ. We use suspension types of the form σ susp primarily to delay computation of recursive calls in evaluating fold expressions. This is not necessary for any v : Grammar and typing rules for values. For any value environment θ, it must be that for all x there is σ such that ⊢ θ(x) : σ. The judgment Γ ⊢ eθ : σ closure is defined in Figure 3.  theoretical concerns, but rather practical: without something like this, implementations of standard programs would have unexpected costs. We will return to this when we discuss the operational semantics. We also observe that in this informal treatment, the types are not polymorphic; in our setting, polymorphism and inductive types are orthogonal concerns.
The map F and mapv F constructors are used to define the operational semantics of fold µt.F . The latter witnesses functoriality of shape functors. Informally speaking, evaluation of mapv F y.v ′ into v traverses a value v of type F [δ], applying a function y → v ′ of type δ → σ to each inductive subvalue of type δ to obtain a value of type Figure 5. The operational cost semantics for the source language. We only define evaluation for closures eθ such that ⊢ eθ : σ for some σ, and hence just write eθ. The semantics for fold depends on the semantics for map, which is given in Figure 6. a technical tool for defining this action when F is an arrow shape, in which case the value of type F [δ] is really a delayed computation and hence is represented by an arbitrary expression. Because the definition of map F and mapv F depend on values, we must define them (and their typing) simultaneously with terms. Furthermore, evaluation of mapv ρ→F results in a function closure value that contains a map F expression, and the function closure itself is, as usual, an ordinary λ-expression. This is also the reason that map and mapv are part of the language, rather than just part of the metalanguage used to define the operational semantics. They are not intended to be used in program definitions though, so we make the following definition: Definition 1 (Core language). The core language consists of the terms of the source language that are typeable not using map or mapv.
The operational cost semantics for the language is defined in Figure 5 and its dependencies, which define a relation eθ ↓ n v, where e is a (well-typed) expression, θ a value environment, v a value, and n a non-negative integer. As with closure values, we write a closure with expression e and value environment θ as eθ, and opt for this notation for compactness (a more typically presentation might be θ ⊢ e ↓ n v). The intended meaning is Figure 6. The operational semantics for the source language map and mapv constructors. Substitution of values is defined in Figure 7. that under value environment θ, the term e evaluates to the value v with cost n. A value environment that needs to be spelled out will be written {x 0 → v 0 , . . . , x n−1 → v n−1 } or more commonly { x → v}. We write θ{y → v} for extending a value environment θ by binding the (possibly fresh) variable y to v. Value environments are part of the language, so when we write eθ, the bindings are not immediately applied. However, we use a substitution notation {v/y} for defining the semantics of mapv t because this is defined in the metalanguage as a metaoperation. Using explicit environments, rather than a metalanguage notation for substitutions, adds a certain amount of syntactic complexity. The payoff is a semantics that more closely reflects typical implementation. Our approach to charging some amount of cost for each step of the evaluation, where that amount may depend on the main term former, is standard. Recurrence extraction is parametric in these choices. We observe that our environment-based semantics permits us to charge even for looking up the value of an identifier, something that is difficult to codify in a substitution-based semantics. Our particular choice to charge one unit of cost for each unfolding of a fold, and no cost for any other form, is admittedly ad-hoc, but gives expected costs with a minimum of bookkeeping fuss, especially when it comes to the semantic interpretations of the recurrences. Another common alternative is to define a tick operation tick : α → α which charges a unit of cost, as done by Danielsson [2008] and others. This requires the user to annotate the code at the points for which cost should be charged, which increases the load on the programmer, but allows her to be specific about exactly what to count (e.g., only comparisons). It is straightforward to adapt our approach to that setting.
The reason for suspending the recursive call in the semantics of fold is to ensure that typical recursively-defined functions that do not always evaluate all recursive calls still  have the expected cost. For example, consider membership testing in a binary search tree. Typical ML code for such a function might look something like the code in Figure 8a. This function is linear in the height of t, because the lazy evaluation of conditionals ensures that at most one recursive call is evaluated. If we were to implement -member-with a -foldoperator for trees that does not suspend the recursive call (so the step function would have type int × bool × bool → bool), as in Figure 8b, then the recursive calls -r0-and -r1are evaluated at each step, leading to a cost that is linear in the size of t, rather than the height. Our solution is to ensure that the recursive calls are delayed, and only evaluated when the corresponding branch of the conditional is evaluated, so the step function has type int × bool susp × bool susp → bool, in which case the code looks something like that of Figure 8c. This issue does not come up when recursive definitions are allowed only at function type, as is typical in call-by-value languages. However, in order to simplify the construction of models, here we restrict recursive definitions to the use of fold δ (i.e. to structural recursion only, rather than general recursion), which must be permitted to have any result type. Given the complexity of the language, it behooves us to verify type preservation. For this cost is irrelevant, so we write eθ ↓ v to mean eθ ↓ n v for some n. Remember that we our notion of closure includes expressions of any type, so this theorem does not just state type preservation for functions.

The recurrence language
The recurrence language is defined in Figure 9. It is a standard system of predicative polymorphism with explicit type abstraction and application. Most of the time we will elide type annotations from variable bindings, mentioning them only when demanded for clarity. The types and terms corresponding to those of Figure 4 are given in Figure 10. This is the language into which we will extract syntactic recurrences from the source language. A syntactic recurrence is more-or-less a cost-annotated version of a source language program. As we are interested in the value (denotational semantics) of the recurrences and not in operational considerations, we think of the recurrence language in a more call-by-name way (although, as we will see, the main mode of reasoning is with respect to an ordering, rather than equality).
3.1. The cost type. The recurrence language has a cost type C. As we discuss in Section 4, we can think of recurrence extraction as a monadic translation into the writer monad, where the "writing" action is to increment the cost component. Thus it suffices to ensure that C bool = unit + unit Figure 10. Some standard types in the recurrence language corresponding to those in Figure 4. Figure 11. The size order relation that defines the semantics of the recurrence language. The macro F [(y : ρ).e ′ , e] is defined in Figure 12.
is a monoid, though in our examples of models, it is usually interpreted as a set with more structure (e.g., the natural numbers adjoined with an "infinite" element).
3.2. The "missing" pieces from the source language. There are no suspension types, nor term constructors corresponding to let, map, or mapv. We are primarily interested in the denotations of expressions in the recurrence language, not in carefully accounting for the cost of evaluating them. Of course, it is convenient to have the standard syntactic sugar let x 0 = e 0 , . . . , x n−1 = e n−1 in e for e{e 0 , . . . , e n−1 /x 0 , . . . , x n−1 }. Because of the way t[(r : ρ).e ′ , e] = e ′ {x → e} τ 0 [(y : ρ).e ′ , e] = e in which the size order is axiomatized, this must be defined as a substitution, not as a βexpansion. Likewise, we still need a construct that witnesses functoriality of shape functors, but it suffices to do so with a metalanguage macro F [(x : ρ).e ′ , e] that is defined in Figure 12. 3.3. Datatype constructor, destructor, and fold. The constructor, destructor, and fold terms are similar to those in the source language, though here we use the more typical type for fold δ . Since type abstraction and application are explicit in the recurrence language, it may feel a bit awkward that β-reduction for types seems to change these constants; for example, (Λα.· · · fold α list e ′ of x.e · · ·) σ would convert to · · · fold σ list e ′ of x.e · · ·. The right way to think of this is that there is really a single constant fold that maps inductive types to the corresponding operator-in effect, we write fold δ for fold δ, and so the substitution of a type for a type variable does not change the constant, but rather the argument to fold.
The choice as to whether to implement datatype-related constructs as term formers or as constants and whether they should be polymorphic or not is mostly a matter of convenience. The choice here meshes better with the definitions of environment models we use in Section 6, but using constants does little other than force us to insert some semantic functions into some definitions. However, one place where this is not quite the whole story is for fold δ , which one might be tempted to implement as a term constant of type ∀ αβ.(F [β] → β) → δ → β, where α = ftv(δ). The typing we have chosen is equivalent with respect to any standard operational or denotational semantics. However, our denotational semantics will be non-standard, and the choice turns out to matter in the model construction of Section 7.4. There, we show how to identify type abstraction with size abstraction in a precise sense. Were we to use the polymorphic type for fold δ , then even when δ is monomorphic, fold δ would still have polymorphic type (for the result), and that would force us to perform undesirable size abstraction on values of the monomorphic type.
3.4. The size order. The semantics of the recurrence language is described in terms of size orderings ≤ τ that is defined in Figure 11 for each type τ (although the rules only define a preorder, we will continue to refer to it as an order). The syntactic recurrence extracted from a program of type σ has the type C × σ . The intended interpretation of σ is the set of sizes of source language values of type σ. We expect to be able to compare sizes, and that is the role of ≤ σ . Although ≤ C is more appropriately thought of as an ordering on costs, general comments about ≤ apply equally to ≤ C , so we describe it as a size ordering as well to reduce verbosity. 3 The relation ≤ C just requires that C be a monoid (i.e., have an associative operation with an identity). In particular, there are no axioms governing 1 needed to prove the bounding theorem; it is not even necessary to require that 0 = 1 or even 0 ≤ C 1.
Let us gain some intuition behind the axioms for ≤ σ . On the one hand, they are just directed versions of the standard call-by-name equational calculus that one might expect. In the proof of the Syntactic Bounding Theorem (Theorem 5.4), that is the role they play. But there is more going on here than that. The intended interpretation of the axioms is that the introduction-elimination round-trips that they describe provide a possibly less precise description of size than what is started with. It may help to analogize with abstract interpretation here: an introductory form serves as an abstraction, whereas an elimination form serves as a concretization. In practice, the interpretation of products, sums, and arrows do not perform any abstraction, and so in the models we present in Section 6, the corresponding axioms are witnessed by equality (e.g., for (β × ), e i = π i (e 0 , e 1 ) ). That is not the case for datatypes and type quantification, so let us examine this in more detail.
Looking forward to definitions from Section 4, if σ is a source language type, then σ is the potential type corresponding to σ and is intended to be interpreted as a set of sizes for σ values. It happens that σ list = σ list, so a σ list value vs = [v 0 , . . . , v n−1 ] is extracted as a list of σ values, each of which represents the size of one of the v i s. Hence a great deal of information is preserved about the original source-language program. But frequently the interpretation (the denotational semantics of the recurrence language) abstracts away many of those details. For example, we might interpret σ list as N (the natural numbers), nil as 0, and cons as successor (with respect to its second argument), thereby yielding a semantics in which each list is interpreted as its length (we define two such "constructor-counting" models in Sections 7.2 and 7.3). Bearing in mind that σ list = µt.F with F = unit + σ × t, the interpretation of F [ σ list] must be unit +( σ ×N). Let us assume that + and × are given their usual interpretations (though as we will see in Section 7, that is often not sufficient). For brevity we will write c and d for c σ list and d σ list . Thus c(y, n) = 1 + n represents the size of a source-language list that is built using c σ list when applied to a head of size y (which is irrelevant) and a tail of size n. The question is, what should the value of d(1 + n) be? It ought to somehow describe all possible pairs that are mapped to 1 + n by c. Ignoring the possibility that it is an element of unit (which seems obviously wrong), and assuming the N-component ought to be n, no one pair (y ′ , n) ∈ σ ×N seems to do the job. However, if we assume the existence of an maximum element ∞ of σ , then (∞, n) is an upper bound on all pairs (y ′ , n) such that c(y ′ , n) = 1 + n, and so it seem reasonable to set d(1 + n) = (∞, n). But in this case, the round trip is not an identity because d(c(y, n)) = (∞, n) ≥ (y, n), and so (β δ ) and (β δfold ) are not witnessed by equality.
Turning to type quantification, the standard interpretation of ∀α.σ is U ∈U σ {α → U } for a suitable index set U (in the setting of predicative polymorphism, this does not pose any foundational difficulties), and the interpretation of a polymoprhic program is the U-indexed tuple of all of its instances. Let λxs.e : α list → ρ be a polymorphic program in the source language. The recurrence extracted from it essentially has the form Λα.λxs.E : ∀α.α list → C × ρ . Let us consider a denotational semantics in which σ list = N × N, where (k, n) describes a σ list value with maximum component size k and length n (this is a variant of the semantics in Section 7.3). We are then in the conceptually unfortunate situation that the analysis of this polymorphic recurrence depends on its instances, which are defined in terms of not only the list length, but also the sizes of the list values. Parametricity tells us that the list value sizes are irrelevant, but our model fails to convey that. Instead, we really want to interpret the type of the recurrence as N → ( C × ρ ), where the domain corresponds to list length. This is a non-standard interpretation of quantified types, and so the interpretations of quantifier introduction and elimination will also be non-standard. In our approach to solving this problem, those interpretations in turn depend on the existence of a Galois connection between N and N×N, for example mapping the length n (quantified type) to (∞, n) (an upper bound on instances), and we might map (k, n) (instance type) to n. The round trip for type quantification corresponds to (k, n) → n → (∞, n), and hence (β ∀ ) is not witnessed by equality (we deploy the usual conjugation with these two functions in order to propogate the inequality to function types while respecting contravariance). We describe an instance of this sort of model construction in Section 7.4, although there we are not able to eliminate the U-indexed product, and so the type of the recurrence is interpreted by U ∈U N → ( C × ρ ).

Recurrence extraction
A challenge in defining recurrence extraction is that computing only evaluation cost is insufficient for enabling compositionality, because the cost of f (g(x)) depends on the size of g(x) as well as its cost. To drive this home, consider a typical higher-order function such as map = fn (f, xs) => fold (fn (x, r) => f x :: r) [] The cost of map(f, xs) depends on the cost of evaluating f on the elements of xs, and hence (indirectly) on the sizes of the elements of xs. And since map(f, xs) might itself be an argument to another function (e.g. another -map-), we also need to predict the sizes of the elements of map(f, xs) which depends on the size of the output of f . Thus, to analyze -map-, we should be given a recurrence for the cost and size of f (x) in terms of the size of x, from which we produce a recurrence that gives the cost and size of map(f, xs) in terms of the size of xs. We call the size of the value of an expression that expression's potential, because the size of the value determines what future (potential) uses of that value will cost (use cost would be another reasonable term for potential).
Motivated by this discussion, we define translations · from source language types to complexity types and · from source language terms to recurrence language terms so that if e : σ, then e : C × σ . In the recurrence language, we call an expression of type σ a potential and an expression of type C × τ a complexity. We abbreviate C × τ by τ . The first component of e is intended to be an upper bound on the cost of evaluating e, and the second component of e is intended to be an upper bound on the potential of e. The weakness of the size order axioms only allows us to conclude "upper bound" syntactically (hence the definition of the bounding relations in Figure 16), though one can define models of the recurrence language in which the interpretations are exact. The potential of a typelevel 0 expression is a measure of the size of that value to which it evaluates, because that is how the value contributes to the cost of future computations. And as we just described, the potential of a type-ρ → σ function f is itself a function from potentials of type ρ (upper Figure 13. The complexity and potential translation of types. Remember that although we have a grammar for structure functors F , they are actually just a subgrammar of the small types, so we do not require a separate translation function for them. bounds on sizes of arguments x of f ) to complexities of type σ (an upper bound on the cost of evaluating f (x) and the size of the result).
Returning to map : (ρ → σ) × ρ list → σ list, its potential should describe what future uses of map will cost, in terms of the potentials of its arguments. In this call-by-value setting, the arguments will already have been evaluated, so their costs do not play into the potential of map (the recurrence that is extracted from an application expression will take those costs into account). The above discussion suggests that (ρ → σ) × ρ list → σ list ought to be ( ρ → C × σ) × ρ list → C × σ list . For the argument function, we are provided a recurrence that maps ρ-potentials to σ-complexities. For the argument list, we are provided a (ρ list)-potential. Using these, the potential of map must give the cost for doing the whole map and give a (σ list)-potential for the value. This illustrates how the potential of a higher-order function is itself a higher-order function.
Since ρ has as much "information" as ρ, syntactic recurrence extraction does not abstract values as sizes (e.g., we do not replace a list by its length). This permits us to prove a general bounding theorem independent of the particular abstraction (i.e., semantics) that a client may wish to use. Because of this, the complexity translation has a succinct description. For any monoid (C, +, 0), the writer monad [Wadler, 1992] )) The monad laws follow from the monoid laws for C. Thinking of C as costs, these say that the cost of return(e) is zero, and that the cost of bind is the sum of the cost of E 1 and the cost of E 2 on the potential of E 1 . The complexity translation is then a call-byvalue monadic translation from the source language into the writer monad in the recurrence language, where source expressions that cost a step have the "effect" of incrementing the cost component, using the monad operation incr(E : C) : C × unit := (E, ( )).
We write out the translation of types in Figure 13 and the recurrence extraction function explicitly in Figure 15. There is a certain amount of notation involved, which we summarize in Figure 14. Recurrence extraction is defined only for typeable terms and only for terms in the core language (Definition 1). τ Potential translation of types. τ Recurrence translation of types. e Recurrence extraction of expressions.
. Notation related to recurrence language expressions and recurrence extraction. Figure 15. The recurrence extraction function on source language terms. On the right-hand sides, (c, p) = e and (c i , p i ) = e i (note that e is always a pair).
For an ordinary function type σ 0 → σ 1 , the translation σ 0 → σ 1 i.e. σ 0 → C × σ 1 includes a cost component in the codomain. In contrast, a polymorphic function type ∀α.τ is translated to ∀α. τ , which does not include a cost component. The reason for this discrepancy is that polymorphic functions in the source language are introduced by let x = e ′ in e, which evaluates e ′ to a value before binding x to a polymorphic version of that value. Thus, the elements of a polymorphic function type incur no immediate cost when they are instantiated (at an occurrence of a variable).
Our first order of business is to verify that recurrences extracted from terms in the source language are themselves typeable in the recurrence language. For a source-language context Γ = x 0 : τ 0 , . . . , x n−1 : τ n−1 , write Γ for x 0 : τ 0 , . . . , x n−1 : τ n−1 . For both the source and recurrence languages, we do not explicitly notate the free type variables of a typing derivation. However, the intended invariant of the translation is that a source language derivation Γ ⊢ e : τ with free type variables α is translated to a recurrence language derivation Γ ⊢ e : σ that also has free type variables α.
Proof. See Appendix B.

The bounding relation and the syntactic bounding theorem
We now turn to the bounding relation, which is a logical relation that is the main technical tool that relates source programs to recurrences. In this section we will refer to source and recurrence language programs extensively, and so we will adopt the convention that E, E ′ , etc. are metavariables for recurrence language terms. The bounding relation eθ σ E is defined in Figure 16. and is intended to mean that E c is a bound on the evaluation cost of eθ and E p is a bound on the value to which eθ evaluates. Bounding of values is defined by an auxiliary relation v val σ E. This latter relation morally should be defined by induction on σ, declaring that a value is bounded by a potential if its components are bounded by corresponding computations on that potential. Of course, function values are defined in terms of arbitrary expressions, and so val ρ→σ must be defined in terms of σ . The standard way to do so for a logical relation is to declare that λx.e is bounded as a value by E if whenever a value v ′ is bounded as a value by E ′ , e{v ′ /x} is bounded as an expression by E E ′ , and we adapt that same idea to our setting here. A naive approach to defining val δ for δ = µt.F would have us define v val δ E in a way that depends on val F [δ] , which is not a smaller type. If we did not permit arrows in shape functors, we could get around this by counting δ-constructors in v. Instead we must take a more general approach. In Figure 17 we define by induction on the structure function F the relations F,ρ and val F,ρ that correspond to bounding at type F [ρ]. We then define val µt.F in terms of val F,µt.F . The source language permits evaluation of closures with open type (in particular, when evaluating a let-binding), so the bounding relation is phrased in terms of open types. Value bounding at open type is defined in terms of all of its instances by closed monomorphic types-we do not enforce any parametricity properties here. Because source language type contexts assign type schemes to identifiers, the standard approach of extending a logical relation on closed terms to open terms by substituting related values requires us to also define a notion of value bounding at type schemes, and we again take this to be in terms of instances at closed types.
We present the relations as a formal derivation system of an inductive definition, because the proofs of Lemmas C.1 and C.2 (technical lemmas needed for the proof of Theorem 5.4, the bounding theorem) rely on a well-founded notion of subderivation. A least relation closed under these rules (which contain a negative occurrence of the relation being defined in the → rule) exists because the type subscript gets smaller in all bounding premises ( or val , not ≤, which is the previously defined size relation on recurrence language Expression bounding at open (monomorphic) types:

Value bounding at open (monomorphic) types:
For all closed ρ: v val σ{ρ/α} E{ ρ /α} v val σ E Value bounding at closed (monomorphic) types: terms). The premise types are smaller for an ordering that considers all substitution instances τ {ρ/α} of τ with a closed monomorphic type ρ to be smaller than the polymorphic type ∀α.τ or a type with a free variable α.τ ; this ordering is sufficient because of the restriction to predicative polymorphism. Although the derivations are infinitary as a result of the clauses corresponding to arrow types and shapes, it is straightforward to assign ordinal ranks to derivations so that the rank of any derivation is strictly larger than the rank of any of its immediate subderivations, justifying such a proof by induction on derivations. The (value) bounding relations in Figures 16 and 17 are really defined on typing derivations. That is, we really define the relations The following lemma acts as an inversion theorem for the bounding relation at inductive types. (1), so it suffices to prove the latter, which is done by a straightforward induction on shape functors.
The bounding relations on closures are extended to (open) terms in the standard way for logical relations. Definition 2 (Bounding relation).
(1) Let θ be a Γ-environment and Θ a Γ -environment. We write θ val Γ Θ to mean that for all x ∈ dom Γ, θ(x) val Γ(x) Θ(x) (note that Γ(x) is a type scheme, so this relation is value bounding at a type scheme).
The main theorem is analogous to the fundamental theorem for any logical relation: every source language program is related (bounded by) the syntactic recurrence extracted from it. The proof is somewhat technically involved, but at its core follows the reasoning typical in the proof of any such fundamental theorem, so we delegate it to the Appendix.
Theorem 5.4 (Syntactic bounding theorem). If e is in the core language and Γ ⊢ e : σ, then e σ e .
Proof. See Appendix C.

Environment models
The syntactic bounding theorem tells us that the syntactic recurrences extracted from source programs provide bounds on the evaluation cost and potential of those programs. However, the syntactic recurrences maintain sufficient information about the source program to describe cost and potential in terms of almost any notion of size. In particular, a syntactic recurrence extracted from a program over an inductive type maintains all the structure of the values of that type-e.g., a syntactic recurrence over a list program describes the bounds in terms of lists again. It is by defining a denotational semantics for the recurrence language that we obtain a "traditional" recurrence, because that permits us to abstract inductive values to some notion of size. We might define a semantics in which a σ tree type is interpreted by the natural numbers N, with the constructor interpreted in terms of either the maximum function (so a tree is interpreted by its height) or the sum function (so a tree is interpreted by its size). So a semantic value in unit + σ × N × N, the one-step unfolding of the interpretation of the tree type, tells us the sizes of the data supplied to the tree constructor. The constructor tells us the size of the tree constructed from such data, and the destructor tells us about the kind of data that can be used to construct a tree of a given size. The denotation of the recurrence extracted from a source program f is then a function T such that T (n) is (a bound on) the cost and size of the result of f (x) when x has size at most n. In other words, the end goal is a "semantic" recurrence obtained by composing a denotation function with the extraction function. Soundness of the denotational semantics with respect to the size ordering in conjunction with the syntactic bounding theorem ensures that the semantic recurrence also provides bounds on the cost and potential of the source program in terms of the potentials of its arguments.
To that end, we need to define an appropriate notion of model for the recurrence language. We will define environment (Henkin) models following [Mitchell, 1996, Ch. 9.2.4], which in turn follows Bruce et al. [1990], specializing the definition to the setting of the recurrence language. Since the recurrence language is characterized by the size order, we require that types be interpreted by preorders, and what would usually be equations describing various semantic functions will be inequalities. This leads to a slight challenge in extending an interpretation of inductive type constructors and destructors to a canonical interpretation of fold δ , because the interpretation of δ is no longer an initial algebra. However, we shall see that it is sufficient to have a initiality condition that is weak (requires existence, but not uniqueness) and lax (is an inequality, not an equality), and that we can arrange.
Applicative structures (and hence pre-models and models) are defined in terms of preordered sets. In such a setting, it is natural to restrict ourselves to functions that respect the pre-order structure-i.e., monotone functions. So in the remaining sections, when A and B are preordered sets, we write A → B for the set of monotone functions from A to B, and A ⇀ B for the set of partial monotone functions from A to B. A → B is preordered pointwise, and id A : A → A is the identity function (we drop the subscript when clear from context). We also frequently write λ λa. · · · for the semantic function that takes a to · · · . 6.1. Models of the recurrence language. We start by defining the notions of type frame and applicative structure for the recurrence language.
Definition 3. A type frame is specified by the following data: • A set U sm of small semantic types and a set U lg of large semantic types with U sm ⊆ U lg ; Figure 18. The denotation (partial) function of types and type schemes into a type frame.
Let TyVar be the set of type variables and let η : TyVar → U sm . The denotation of τ with respect to η, τ η, is given in Figure 18.
Definition 5. An applicative structure is specified by the following data: • Distinguished elements 0, 1 ∈ D U C and an associative function + : D U C → D U C such that 0 is a right-and left identity for +.
such that TyApp • TyAbs) ≥ id. Note that TyAbs is a partial function. Remember that when we write, e.g., D U → D V , we mean the monotone functions from D U to D V , and hence all of the semantic functions that make up the data of an applicative structure are monotone.
For an applicative structure and environment η, define a partial denotation function Γ ⊢ e : σ η as in Figure 19. The only way in which · · may fail to be total is if the arguments to Abs or TyAbs are not in the corresponding domains (because we start with a type model, we know that µ and ∀ are only applied to functions in their domains).
Definition 6. Let U be an applicative structure.
The indirection of interpreting syntactic types by semantic types, and then interpreting terms of a given syntactic type as elements of a domain associated to the corresponding semantic type is necessary, especially in our setting of non-standard models. This makes is much easier (seemingly, possible) to define things like the µ operator. Without the indirection, we would have to define µ on (functions on) a collection of domains, some of which represent syntactic types. That ends up being very difficult to do. For example, we might have to first define a notion of polynomial function on the semantic domains in order to define the domain of µ, and then somehow identify each semantic polynomial function with a structure functor. But doing so gets us into problems with unique representation; e.g., there may be multiple structure functors corresponding to the same semantic polynomial. And with non-standard models, we seem to have even more troubles, because we end up trying to define the interpretations of inductive types simultaneously with the µ function. But first interpreting the syntactic types by semantic types gives us a way around these problems, because (if we wish) we can define the semantic types to be closely tied to the syntactic types. That is exactly what we do for the standard type frame, so we can essentially define µ syntactically, and then choose a domain corresponding to µt.F (which is a semantic type as well as a syntactic one) after having defined µ. Figure 19. The denotation (partial) function into an applicative structure. For constructors and destructors, assume δ = µt.F and fv(δ) = {α 0 , . . . , α n−1 }, and define η * = η{ α → U }.
Proposition 6.2 (Environment model soundness). If U is an pre-model, then U is a model.
Proof. By induction on the derivation of Γ ⊢ e ≤ τ e ′ .
One might hope that a model of the fragment of the recurrence language that omits fold δ can be extended to one that does, but in our setting this does not quite hold. Since we only have directed versions of the usual equalities, initial algebras for structure functors may not exist. And even if they do, they are not necessarily what we want. For clarity, in this discussion we will write syntactic types for semantic types. The point behind different semantics is to abstract inductive values to some notion of size, and when this abstraction is non-trivial, D µt.F and D F [µt.F ] are probably not isomorphic. Instead of the usual initial algebra for interpreting µt.F , we typically want an algebra C F : D F [µt.F ] → D µt.F such that for any other algebra s : where Map F is a semantic function that corresponds to the F [·, ·] macro. Relative to the usual definition of initial algebra, this requirement is weak, in that we ask only for existence of a Fold function making the diagram commute (β reduction) and not the uniqueness of Fold (η/induction), and it is lax, in that we ask that β-reduction holds only as an inequality, rather than an equality. Nonetheless, under assumptions that turn out to be relatively easy to ensure, we can define Fold Φ,U . Given a subset X ⊆ A of a preordered set A, we say that a ∈ A is a least upper bound of X, written a = X, if for all x ∈ X, x ≤ a, and if b ∈ A satisfies the condition that for all x ∈ X, x ≤ b, then a ≤ b. When A is preordered, X may not exist, and when it does, it need not be unique. If A is a partial order (i.e., a ≤ b ≤ a implies that a = b), then X is unique when it exists, and we say that A is a complete upper semi-lattice if A is a partial order and X exists for every X ⊆ A. Though this seems like a very strong condition, in practice it is easy to ensure.
In a model in which every D U is a complete upper semi-lattice, we would like to define A priori, this definition may not be well-founded, but in fact it is, as shown in the next proposition.
Proposition 6.3. Suppose that U = {D U } U ∈U lg is a model of the fragment of the recurrence language that omits fold δ and each D U is a complete upper semi-lattice, and suppose that Fold is defined by (std-fold). Then: (1) For all s, Fold Φ,U s is total and monotone.
(1) Fix s and consider (2) Totality follows from (1) and monotonicity from the fact that the function that maps a monotone function to its least fixed point is itself monotone.
The proof of Prop. 6.3, and hence the interpretation of fold δ , may seem a bit heavyhanded, making use of general least fixed point theorems and even iterating into the transfinite. As we noted earlier, we are in a setting in which we do not have (and do not want) initial algebras, but must nonetheless show an initiality-like property of a given algebra. Accordingly, we would expect to use technology at least as strong as that needed for typical initial algebra existence theorems. The canonical such theorem (e.g., as described by Aczel [1988, Thm. 7.6]) verifies that the least fixed point of a set-continuous operator is an initial algebra, and the verification consists of constructing the equivalent of Fold s by induction on the (ordinal-indexed) construction of the least fixed point.
The reader may have noticed that an alternative possible definition for Fold is and Prop. 6.3 would still hold. This fact witnesses that the initiality condition for C Φ : .
Monotonicity of f only allows us to conclude that f ( X) ≥ {f (x) | x ∈ X}, but this putative definition for Fold s exposes a case in which this inequality is strict.
6.2. The standard type frame. Our last step in our general discussion of models is to define the type frame upon which all of our examples will be based. It gives us enough data to provide a standard definition of the functions Φ U,V , which in turn lets us use (std-fold) to define Fold and so for most of our examples, it will suffice to define C Φ (because we will Our examples are all based on variations of the standard type frame, which is defined as follows: • U sm is the set of closed types and U lg the set of closed type schemes of the recurrence language. • →, ×, and + are the standard type constructors; e.g., σ 0 + σ 1 = σ 0 + σ 1 .
Proposition 6.4. The standard type frame is a type model.
For any applicative structure based on (an extension of) the standard type frame, define ) for each closed structure functor F and closed ρ and σ by: Lemma 6.5. If U is an applicative structure based on an extension of the standard type frame that is a model for the fragment of the recurrence language that omits fold δ , Γ, y : ρ ⊢ e ′ : σ, Γ ⊢ e : F [ρ], and η is a Γ-environment, then Proof. By induction on F .
Combining Prop. 6.3 with Lemma 6.5, we conclude that to define a model of the recurrence language, it suffices to define an extension of the standard type frame and the following applicative structure data: • The sets D τ , along with an argument that D τ is a complete upper semi-lattice; • The semantic functions for arrow, product, and sum types; • C F for each structure functor F . From this data we can define D F (x) = {z | C F ≤ x}, F ρ,σ as just given, and Fold F,σ by (std-fold). Of course, there are models that are not constructed this way; Section 7.5 gives an example that is useful for extracting recurrences for lower bounds.
6.3. Syntactic sugar. We now introduce some syntactic sugar that will make our discussion of recurrences somewhat more pleasant. To simplify the discussion, we restrict the details to the source language type σ tree and its recurrence language potential σ tree, but we will use analogous notation for other datatypes such as nat and α list in our examples. Many of our source-language functions are really structural folds over some standard datatype-that is, the step function is a case expression where the argument for each branch is really the argument to one of the datatype constructors. Accordingly, we introduce notation for such fold expressions: for y / ∈ fv(e emp ) ∪ fv(e node ), fold σ tree e of emp ⇒ e emp | node ⇒ (x, r 0 , r 1 ).e node is syntactic sugar for fold σ tree e of w.case w of y.e emp ; y.e node {π 0 y, π 1 y, π 2 y/x, r 0 , r 1 }. We introduce a similar notation in the recurrence language: fold σ tree e of {emp ⇒ e emp | node ⇒ (x, r 0 , r 1 ).e node } is syntactic sugar for fold σ tree e of (w : F σ tree [ρ]).(case w of y.e emp ; y.e node {π 0 y, π 1 y, π 2 y/x, r 0 , r 1 }) where w and y are fresh variables.
It would be nice to establish an identity of the form fold σ tree · · · = fold σ tree · · · , but the size-order axioms, which give us only inequalities, are too weak. However, the models that we will consider validate many equations, so we can set out a nice relationship. In the following proposition, we say "in the semantics, e = e ′ " to mean that for any η, e η = e ′ η: Proposition 6.6. Suppose that we have a model such that • In the semantics: if e ′ c = 0, then e{e ′ /x} = e { e ′ p /x}; and • In the semantics: c + c case e of {x.e i } i=0,1 = case e of {x.c + c e i } i=0,1 .
If Γ ⊢ fold σ tree e of emp ⇒ e emp | node ⇒ (x, r 0 , r 1 ).e node : ρ, then in the semantics, While the models that we discuss in subsequent sections satisfy the hypotheses of Prop. 6.6, they are not necessarily satisfied in an arbitrary model. That requires additional axioms that correspond roughly to η axioms.
7. Examples 7.1. The standard model. For the standard model, we first extend the standard type frame by including the constant ⊥ in U sm . A semantic type (scheme) is proper if it has no occurrences of ⊥. The proper semantic types (type schemes) correspond exactly to the closed syntactic types (type schemes). In the definitions of µ and ∀, we take F and τ to be proper. We define the sets A σ by induction on σ as follows: 5 • A C = N, the natural numbers.
, and let the semantic functions for arrows, products, and sums be the identity functions. The definitions of C F , D F , and Fold F,σ are based on the standard initialalgebra semantics. Note that we cannot use (std-fold) because the A σ are not complete upper semi-lattices, and hence the hypotheses of Prop. 6.3 do not hold, and hence (β δfold ) must be verified directly.
At first blush, this model is not particularly interesting. There is no abstraction of values to sizes and the "order" on costs is the identity, so the recurrences extracted from source language programs describe the exact cost of those programs in terms of the argument values. However, this is a standard model of (predicative) polymorphism, and so we can hope that parametricity may have some interesting consequences. Free theorems [Wadler, 1989] have been used to obtain relative cost information, and we discuss this further in Section 9. Here, we apply parametricity to the recurrence language and sketch the argument that if g : α list → α list, then the cost of g(xs) depends only on the length of xs (the same can be said for the length of g(xs), but this follows from parametricity applied to the source language). For any ρ, let us define T ρ (xs) = (( g ρ ) p (xs)) c , the exact cost of evaluating g(xs) (since · is a monadic translation and the interpretation of inductive types is the standard one, syntactic values of list type in the source language are isomorphic to the semantic values in the model). The goal is to show that if xs : ρ list and 5 The collection of sets {A σ }σ∈U sm must be contained in some set that contains ∅ and a one-element set and is closed under disjoint unions, products, function spaces, unions of chains, and products indexed by Usm. V ω 1 in the standard set-theoretic hierarchy suffices.
ys : σ list are of the same length, then T ρ (xs) = T σ (ys). To do so, we apply parametricity to λ λρ.λ λxs.T ρ (xs) ∈ A ∀ρ.ρ list→C . We take the relational interpretation of C to be equality (so the cost constants 0 and + preserve the relation). Expanding the definition of parametricity, this means that for any ρ and σ and relation R ⊆ A ρ × A σ , for any xs ∈ A ρ list and ys ∈ A σ list , if R list ⊆ A ρ list × A σ list holds for xs and ys, then the relational interpretation of C holds for T ρ (xs) and T σ (ys). Since the relational interpretation of the cost type is equality, this would give the result, so it suffices to show that there is an R such that (R list)(xs, ys) holds whenever xs and ys have the same length. However, the standard relational lifting R list holds whenever xs and ys have the same length and xs i is related to ys i by R, so taking R to be the total relation achieves this. We conclude that if xs and ys have the same length, then the cost of g(xs) and g(ys) is the same. 7.2. Constructor size and height. We now describe a model in which a value v of inductive type δ is interpreted either by the number of δ constructors in v (constructor size) or by the maximum nesting depth of δ-constructors in v (constructor height), so that it reflects common size abstractions such as list length, tree size, and tree height. For example, in this model, the interpretation of the recurrence extracted from a function with domain σ list describes the cost in terms of the length of the argument list. For concreteness we will define the constructor size model. For the interpretation of the types, we will need two versions of the natural numbers: N ∞ 0 = {0, 1, . . . , ∞} for costs, and N ∞ 1 = {1, 2, . . . , ∞} for sizes of inductive values, which must be at least 1 because every value contains at least one constructor.
The presence of ∞ may be perplexing, since all programs in the source language terminate. However, it is not always possible to give a finite upper bound on cost or potential in terms of the potential of the argument, because the notion of potential used in this model may not identify all possible sources of recursive calls. For example, consider the function sumtree defined in Figure 22 that sums the nodes of a nat tree. The cost and size of sumtree t depend on the size of t and the sizes of its labels, whereas in this model, the potential of t only tells us the former. Since sumtree is definable in our source language, its recurrence can be extracted, and hence must have a meaning in this model; the only sensible interpretation is one that maps every tree size to the trivial upper bound of ∞ for both cost and potential.
We start by extending the standard type frame with additional small types N 0 , N 1 ∈ U sm . Then we define the sets V τ , observing that each V τ is a complete upper semi-lattice. This allows us to construct a model by just defining C F . The sets V τ are defined as follows: • V σ 0 →σ 1 = the set of monotone functions from V σ 0 to V σ 1 with the usual pointwise order, taking Abs and App to be the identity functions. • V σ 0 ×σ 1 = V σ 0 × V σ 1 with the usual component-wise order, taking Pair and Proj to be the standard pairing and projection functions.
, which we define in Section 7.2.1.
• V µt.F = N ∞ 1 . We define C F in Section 7.2.2 (recall that we write C F for C λ λV. F {V /t} , etc., and that we can define D F and Fold F,σ from it).
• V ∀α.τ = σ∈Usm V τ {σ/α} , with the pointwise order, taking TyAbs and TyApp to be the identity functions. Once we define the interpretation of sums and datatypes, it is straightforward to verify that this is a model.
Proof. Since Abs and TyAbs are total, it suffices to verify the conditions on the semantic functions. This is trivial for arrows, products, and type quantification; sums and inductive types are handled in the next two sections. 7.2.1. Interpretation of sums. As we observed, we need to ensure that all the sets V σ are complete upper semi-lattices. Preserving the complete upper semi-lattice property is straightforward for all type constructors except sum. We could take the usual disjoint sum along with a new infinite element ∞ that is a common upper bound of elements on both sides, but that ends up leading to very weak bounds in practice. For example, recall that D σ list (2) should tell us about the data that can be used to construct a list of size ≤ 2 (which is a cons list, because we count the number of c σ list constructors, so nil has size 1). If we were to interpret sums as just proposed, both Inj 0 ( * ) and Inj 1 (a, 0) are such values, and their least upper bound would be ∞. It is not hard to parlay this into an argument that if tail = λxs.case xs of x.nil; x.π 1 x is the usual tail function on σ list, then the recurrence extracted from tail gives a bound of ∞ for all lists of length > 1. While correct, this is hardly satisfying! Instead, we take inspiration from abstract interpretation [Cousot and Cousot, 1977]: we will define D F (n) to be the set of values x such that C F (x) ≤ n. We can arrange this for the typical cases of interest (i.e., finitary inductive datatypes such as lists and trees) by defining V σ 0 +σ 1 to be the downward closed subsets of V σ 0 ⊔ V σ 1 . We could arrange this for all inductive datatypes if we were to do something similar in the interpretation of arrows and products, but that entails some additional notational cost in reasoning about extracted recurrences while providing no benefits for the examples that we present. We start with some standard order-theoretic and set-theoretic definitions: • For any partially ordered set A, the order ideal of A is

O(A) is partially ordered by set inclusion and is a complete upper semi
• If X 0 and X 1 are partially-ordered sets, X 0 ⊔ X 1 is the usual disjoint union with injection functions in i : For the interpretation of sums, we define V σ 0 +σ 1 = O(V σ 0 ⊔ V σ 1 ) with the semantic functions defined by Proof.
Note that OA is a monad on the category of partially ordered sets and monotone functions, with unit A → OA given by ↓ A , and multiplication OOA → OA given by union, and it plays the role of a powerset monad on posets (the ordinary powerset operation, without the additional downward closure requirement, does not have a monotone function A → P(A), because x ≤ A y does not imply that {x} ⊆ {y}. When a partially ordered set is a complete upper semilattice (i.e. supports the maximum operation that we use to interpret the recursor), it is an algebra for this monad, i.e. there is a monotone function OA → A, satisfying some equations. Thus, another way of understanding these models is that, for functions and products, we build algebras 7.2.2. Semantic functions for inductive datatypes. We define C F by first defining a function size F : V F [µt.F ] → N ∞ 0 . For δ = µt.F , a semantic value of type F [µt.F ] represents the data from which a value of type δ is constructed, but with the inductive substructures replaced by their sizes, and size F returns the size of the inductive value constructed from that data.
It is not hard to see that ⊢ t : σ tree = 2n + 1, where n is the usual size of t (i.e., ⊢ t : σ tree is the number of internal and external nodes of t). Since this is linear in the usual notion of size of a tree, it suffices for showing that the recurrences that we extract have the expected O-behavior.
Destructors exhibit the desired behavior; consider σ list again: In other words, a list of size 1 must be nil and a list of length at most x is either nil or cons(x, xs), where xs has length at most x − 1. Remember that For σ tree, the result is equally pleasant: Finally, we observe the following simple forms for the denotation of recurrences over lists and trees: Proposition 7.3.
(1) If f n = fold σ list y of {nil ⇒ e nil | cons ⇒ (x, r).e cons } η{y → n}, then in the constructor size and height models, The second form for f n, n > 1, follows from monotonicity of the denotation function.
In the constructor height model, replace n 0 + n 1 < n with n 0 ∨ n 1 < n.
Although we will primarily use Prop. 7.3, it may be instructive to work through an example of explicitly constructing Fold F,ρ from the proof of Prop. 6.3. Consider F = unit+t (the structure functor for nat), and set s(x) = Case(x, λ λu.1, λ λu.1 + x). s might be the step function for the recurrence that describes the cost of the copy function on nat. Define Q as in the proof of Lemma 6.3. In this setting, the bottom element at which we start iterating Q is the function that is constantly 0. Set f 0 = Q ⊥ and f n+1 = Q f n . Just as in the calculation of D F σ list , | node ⇒ (x, r 0 , r 1 ).node(x, force r 0 , force r 1 ).
and so a bit more calculation shows that f k (n) = n, n ≤ k + 1 k + 1, n > k + 1.
It is not hard to see that f ω (n) = n is a fixed point of Q, so we conclude that Of course, this is precisely what we expect, though for readers familiar with how a typical recursive function on numbers is defined by successive approximations, the route may feel a bit different. Usually when defining a recursive function on numbers, one takes the flat order and starts with the everywhere-undefined function. For a typical total function, the k-th approximation is a partial function that is defined and correct on some initial segment of the natural numbers and undefined elsewhere. Here we take the (more-or-less) standard order and start with a function that is everywhere an unlikely bound (namely, 0). Each successive approximation yields a function with more likely bounds, terminating with a (hopefully low but) correct bound. In the case of a partial recursive function, the "bad" case is that the function is not defined for some numbers (the value of the approximants never gets above ⊥). In our setting, the "bad" case is that the bound is infinite (the value of the approximants never stops growing). The reader may wish to compare this with the use of N ∞ 0 by Rosendahl [1989], where ∞ corresponds to the bottom element in the usual CPO semantics for fixpoints. We return to this in Section 8 when we discuss general recursion. 7.2.4. Example: tree copy. For a first "sanity check," let us analyze the tree copy function that is defined in Figure 20. We will also describe some of the main features in the analysis that are typical of all of our examples. The first is that a source language program e = λx, y, z.e ′ extracts to a recurrence of the form (0, λx.(0, λy.(0, λz. e ′ ))). However, we are really only interested in e ′ as a function of the potentials x, y, and z. Accordingly, when analyzing a program such as e, we focus on the recurrence language program λx, y, z. e ′ . Here, this means we will analyze the (denotation of the) recurrence copy σ tree that is also shown in Figure 20. Second, we shall use Prop. 6.6 freely as though it is a theorem about the syntax when we write our examples. Third, in our examples, we typically use the identifier r in syntactic recurrences for a recursive call to the computation of a complexity, and hence r p and r c correspond to recursive calls that compute potential and cost, respectively. Finally, we remind the reader that our goal is to show that the semantic recurrences are essentially the same as those that we expect to arise from an informal analysis, and so we make no attempt to solve them.
The analysis for copy σ tree proceeds as follows. Define T (n) = ( copy σ tree (n)) c . Following the definition of the denotation function and using Prop. 7.3 and facts about ∨ and mem = λcmp σ×σ→order , t σ tree , x σ .
7.2.5. Example: binary search tree membership. For an interesting example, let us consider membership testing in σ-labeled binary search trees. First we define the type order = unit + unit + unit and write case e of LT.e 0 ; EQ.e 1 ; GT.e 2 for case order e of x.e 0 ; x.e 1 ; x.e 2 , and we assume comparable notation in the recurrence language. The membership test function is given in Figure 21.
Let us consider an informal analysis of mem, which is somewhat simpler to describe in reference to the member function of Figure 8(a). Let T (h) be the number of calls to member in terms of the height of t. We would probably argue that T (1) = 1 and for h > 1, cost of a case is bounded by the costs of its branches , where h 0 , h 1 < h. But since the only information we have is that t 0 and t 1 are subtrees of some tree t of height h, what we must really mean is that so this is the recurrence we expect to see in a formal analysis.
Taking the same approach as in the previous section, we analyze the recurrence mem given in Figure 21, this time considering its denotation in the constructor height model. The extracted recurrence makes explicit the dependence of the complexity of mem on the complexity of the comparison function cmp. Of course, a typical analysis will make assumptions about this complexity. The most common such (and the one we implicitly made in our informal analysis) is that the cost of the comparison function is independent of the size of its arguments, which we can model here by assuming that (cmp (x, y)) c = 0 for all x and y (more precisely, we only analyze mem cmp under the assumption that cmp satisfies this condition). Define T (h) = ( mem cmp h x) c and assume that cmp(x, ∞) = A ⊔ B ⊔ C. Then making use of Prop. 7.3, Again, we have essentially the same recurrence as given by the informal analysis. The last inequality is valid because A, B, and C are all subsets of { * }, and hence are either ∅ or { * } itself, and we take advantage of the fact that ∅ = 0. The comparison with ∞ might be a bit perturbing. In this model, the labels do not contribute to the potential of a tree. Since the comparison in the recurrence arises from the comparison of x with an arbitrary node label y, the best we can say about the potential of y is that it is at most ∞. For another perspective, keep in mind that cmp is monotone, so unless it is a particularly odd function, cmp(x, ∞) = { * } ⊔ { * } ⊔ { * }, which forces the recurrence to take all possible outcomes into account. This is precisely what we would expect in an informal analysis.
7.2.6. Inductive types as an abstract interpretation. Our justification for the interpretation of sum types appealed to intuition from abstract interpretation. For datatypes with structure functors that are sums of products (e.g., lists and trees), the connection goes beyond just intuition, as it is easy to see that not only do we have that D • C ≥ id (β δ ) but also that C • D = id. This is precisely the kind of Galois connection we would expect to see in an abstract interpretation, where here we think of the datatype as being the abstract domain and its unfolding to be the concrete domain. 6 Intuitively, this is exactly how we think of models of the recurrence language as performing a size abstraction on datatypes. Interpreting a datatype value (i.e., an application of the constructor) as a size abstracts away information. Destructing a size tells us how a value of that size may be constructed from other data, but that data can only tell us the sizes of the substructures used in the construction. In other words, the application of the destructor gives us more concrete information about a size, namely, something about the composition of a value of that size.
7.3. Counting all constructors. The cost of some functions cannot be usefully described in terms of the "usual" notion of size captured by the model V of the previous section.
For example, to usefully analyze the sumtree function of Figure 22, we need a model in which the size of a nat tree value measures both the number of nat tree constructors and the number of nat constructors. In this section, we give an example of how to construct such a model. In it, a value v of inductive type is interpreted by a function φ such that φ(δ) is the size of the largest maximal subtree of v that contains only δ-constructors. For v : nat tree, this means that v (nat tree) is the usual size of v, v (nat) is the maximum label size of v, and v (δ) = 0 for δ / ∈ {nat tree, nat}. Because we want to distinguish between constructors for different inductive types, it is convenient to use the following alternative grammar for types and structure functors, which just spells out the closed type production for structure functors: The content of the next proposition is just that the grammar ( * ) defines the same words as that of Figure 9.
(1) If σ is a type by the grammar ( * ), then σ is a structure functor by the grammar ( * ).
(2) σ is a type by the grammar of Figure 9 iff σ is a type by the grammar ( * ), and F is a structure functor by the grammar of Figure 9 iff F is a structure functor by the grammar ( * ). Proof.
(2) Induction on the µ-nesting depth of σ and F . The main idea is that we treat t as a fixed symbol, rather than a meta-variable ranging over a class of variables, so inside the µt.F production of F , it is no longer possible to refer to the "outer" t, and the µt.F production of F always corresponds to a constant shape functor.
The type frame is the same as for the constructor-counting model of Section 7.2; for the current model, we write W σ for the interpretation of σ. Except for inductive types, the clauses for W σ are the same as those for V σ from Section 7.2. Set D = {µt.F | F closed} and • W µt.F = {φ ∈ D → N ∞ 0 | φ(µt.F ) ≥ 1, δ not a syntactic subtype of F ⇒ φ(δ) = 0}.
6 Of course, the domains here are not of finite height as in typical AI analyses, but that is typically for the benefit of computability of those analyses; that would correspond to computing the denotation of the bounding recurrence, which is not our primary concern here. To define C F and D F , we define size F,δ : W F [δ] → (D → N ∞ 0 ) similarly to the previous section. The additional subscript enables us to track which datatype is the "main" datatype, as the counting is different for products for the main datatype and others. The definition is as follows: fold δ is interpreted by (std-fold) as usual.
7.3.1. Example: the potential of nat tree. Although we could prove a general theorem to show that size F,δ encapsulates the description given above, seeing the details of the specific case of nat tree is more illuminating. To start, some notation is helpful: set φ nat n ∈ W nat and φ nat tree n,k ∈ W nat tree to be the functions First we start with a useful lemma: Lemma 7.5. S (φ) = χ Fnat + φ, and in particular, S (φ nat k ) = φ nat k+1 . Proof.

7.3.2.
Example: summing the nodes of a nat tree. Let us use this model to analyze the function sumtree : nat tree → nat that sums the nodes of a nat tree. Its definition is given in Figure 22, along the relevant extracted recurrences. An informal analysis might proceed as follows. Because the cost of sumtree depends on both the cost and size of the result of plus as well as the size of the results of the recursive calls, we must extract recurrences for all of these. If S plus (m, n) and T plus (m, n) are the size of the result and the cost of plus(m − 1, n − 1), respectively (recall from Figure 4 that n is the source language numeral for n), then an informal analysis yields the recurrences Similarly, if S st (n, k) and T st (n, k) are the size of the result and the cost of sumtree(t) when t has maximum label size n and size k, we end up with the recurrences S st (n, 1) = 1 S st (n, k) = {S plus (n, S plus (S st (n, k 0 ), S st (n, k 1 ))) | k 0 + k 1 < k} and T st (n, 1) = 1 To solve these recurrences, one would first use any standard technique to conclude that S plus (m, n) = m + n − 1 and T plus (m, n) = m to simplify the recurrence clauses for S st and T st , then establish bounds on the latter by induction. However, the solution of the recurrences is not our focus here, but rather the justified extraction of them. Now let us turn to our formal analysis. SetS plus (φ, φ ′ ) = ( plus φ φ ′ ) p . Then making use of Prop. 7.6,S plus (φ nat 1 , φ ′ ) = φ ′ and for m > 1, This recursive description ofS plus is sufficient to prove thatS plus (φ nat m , φ ′ ) ≥ φ ′ , and so we can conclude the reasoning with S plus (φ nat m , φ ′ ) = χ Fnat +S plus (φ nat m−1 , φ ′ ) and so in particular , recurrences that are equivalent to those derived informally. The analysis ofT plus (φ, φ ′ ) = ( plus φ φ ′ ) c is similar and results in the recurrencẽ is the bottom element of W nat and we can prove from this recurrence that S st (φ nat tree n,k ) is monotone with respect to n, we can conclude this reasoning with S st (φ nat tree n,k ) = {S plus (φ nat n ,S plus (S st (φ nat tree n,k 0 ),S st (φ nat tree n,k 1 ))) | k 0 + k 1 < k}, which is analogous to the recurrence we derived informally. The analysis ofT st (φ nat tree n,k ) = ( sumtree φ nat tree n,k ) c is similar and leads tõ 1+T st (φ nat tree n,k 0 ) +T st (φ nat tree n,k 1 )+ T plus (S st (φ nat tree n,k 0 ),S st (φ nat tree n,k 1 ))+ T plus (φ nat n ,S plus (S st (φ nat tree n,k 0 ),S st (φ nat tree n,k 1 ))) | k 0 + k 1 < k As a final note, in order to obtain the desired final form, we sometimes had to do some reasoning about the function on the basis of its recurrence, such as proving that the function is monotone. In fact, such reasoning is almost always required in the informal analysis as well, even though we typically gloss over such points when analyzing algorithms.
In may be helpful to contrast this analysis with the interpretation of plus and sumtree in the model of Section 7.2. Since nat values involve no other datatype constructors, the interpretation of plus is essentially just the same, only requiring less notation to write down. However, the cost component of sumtree {n/t} is less helpful. Because the model of Section 7.2 only accounts for the tree constructors, it does not account for the sizes of the node labels, and so this computation includes the cost component of plus x (plus r 0p r 1p ) p {∞, . . ./x, r 0 , r 1 } and this will result in a bound of ∞ (cf. to the occurrence of ∞ in the analysis of mem in the previous section, which did no harm there). This is correct as a bound. It reflects a cost analysis in which we have decided that we are counting each recursive call as a computation step, but then analyze a program in which data values whose size we ignore is the source of some recursive calls. However, this rather poor choice of size for this particular context yields a very weak bound, and so shows more generally that the choice of model does really matter. 7.4. Size abstraction and polymorphism: merging the constructor-counting models. Let us make a couple of observations about the previous two sections. It seems at least intuitive that counting only the main constructors is a more abstract notion of size than counting all constructors. And it also seems that even if we are working in the model of Section 7.3, if we have a polymorphic function in hand, it ought to be analyzable by just counting main constructors. This leads to the idea that if we have a model in hand (such as counting all constructors), then at least in some cases, it ought to be possible to interpret polymorphic recurrences so that the potentials arise from a more abstract notion of size than that given by the model. We give an example of how that might be done now.
are two models of the recurrence language, both based on the (same extension of the) standard type frame. We say that U ′ is an abstraction of U, or U is a concretization of U ′ , if for every σ ∈ U sm there are functions such that for all σ, conc σ is monotone, conc σ • abs σ ≥ id D σ and abs σ • conc σ = id D ′σ .
that is defined as follows: • For σ ∈ U sm , B σ = D σ , with the semantic functions for small types taken from U. (1) If U and U ′ are applicative structures, then U → U ′ is an applicative structure.
As an example, we define abstraction and concretization functions in Figure 23 that show that the main constructor counting model V from Section 7.2 is an abstraction of the all-constructor counting model W from Section 7.3.
(2) By induction on σ; we just do σ = σ 0 + σ 1 . Let us write abs for abs σ 0 +σ 1 , abs i for abs σ i , and similarly for conc. To see that abs • conc = id, notice that (abs and a ∈ Y i such that a ′ ≤ abs i (b) and b ≤ conc i (a), and hence a ′ ≤ abs i (conc i (a)) = a (by monotonicity and the induction hypothesis). But since Y i is downward closed, To see that abs • conc ≥ id, notice that if a ∈ Y 0 ⊔ Y 1 , then a ∈ Y i for some i, and hence a = abs i (conc i (a)) ∈ ↓ abs To see that conc • abs ≥ id, suppose b ∈ X i . Then by the induction hypothesis b ≤ (conc i • abs i )(b), and by unraveling the definition, Proposition 7.9. W → V is a model.
Proof. From Props. 7.7 and 7.8 and the fact that TyAbs V is total.
The definition of the abstraction and concretization functions in Figure 23 looks fairly canonical, so a natural question is whether for any two models of the recurrence language one can extend given functions on the interpretations of base types to all small types. In fact these definitions are an instance of a general pattern, but to state the pattern we will need a few definitions. A 2-category is a generalization of a category with a notion of morphism-between-morphism: if X and Y are objects, and f, g : X −→ Y are morphisms, then we will write f ≤ C g : X −→ Y for a 2-cell from f to g. We will mainly consider the 2category Preorder, whose objects X, Y are preordered sets, whose morphisms f : X −→ Y are monotone functions, and whose 2-cells f ≤ g : X −→ Y are bounds ∀x : X.f (x) ≤ Y g(x). We will also need Preorder op (the 1-cell dual of Preorder): the objects are again preorders, a 1-cell X −→ Preorder op Y in Preorder op is a 1-cell in Y −→ Preorder X, i.e. a monotone function Y → X, but the 2-cells f ≤ P reorder op g : X −→ P reorder op Y are still the 2-cells f ≤ Preorder g : Y −→ Preorder X, i.e. ∀y : Y.f (y) ≤ X g(y). A standard construction is to take the cartesian product of two 2-categories, where the objects, 1-cells, and 2-cells are given pointwise; in particular we will consider Preorder × Preorder and Preorder op × Preorder. A 2-functor F : C → D between 2-categories acts on objects, 1cells (preserving identity and composition either strictly or up to 2-cell isomorphism), and 2-cells. For example, a (strict) 2-functor F : Preorder → Preorder consists of (0) for each preorder X, a preorder F (X); (1) for each monotone function f : I.e. F sends preorders to preorders and monotone functions to monotone functions, in such a way that if g bounds f then F (g) bounds F (f ).
An abstract interpretation in the sense above is often called a Galois insertion, which is a reflection in Preorder: a (strict) reflection of A into C consists of a pair of 1-cells abs ⊣ conc where abs : C → A and conc : A → C, with an equality abs • conc = id A and a 2-cell id C ≤ C conc • abs. A standard observation is that any 2-functor F : C → D preserves reflections (this is used, for example, in domain theory [Smyth and Plotkin, 1982]): if abs ⊣ conc is a reflection then F (abs) ⊣ F (conc) is a reflection between F (C) and F (A). Applying F to the equality abs • conc = id A and using strict preservation of identity and composition gives F (abs) • F (conc) = id F (A) , and using the action on 2-cells of F on id C ≤ C conc • abs (and again preservation of identity and composition) gives This all means that we can lift the abstraction and concretization from base types to any type constructor that extends to a 2-functor. The product of preorders X × Y is the action on objects of a functor Preorder × Preorder → Preorder, where the action on maps f 0 : X 0 → X ′ 0 and g : X 1 → X ′ 1 is given by f 0 × f 1 : X 0 × X 1 → X ′ 0 × X ′ 1 := z → f 0 (π 0 z), f 1 (π 1 z) This acts on 2-cells (preserves bounds) because pairing and application are monotone operations. To show that it preserves composition, we need a full β-reduction equation, and to rev = λxs.let rev ′ = λxs.fold α list xs of nil ⇒ λzs.zs | cons ⇒ (x, r).λzs.(force r)(cons(x, zs)) in rev ′ xs nil rev ′ = Λα.λxs.fold α list xs of nil ⇒ (1, λzs.(0, zs)) | cons ⇒ (x, r).(1, λzs.r c + c r p (cons(x, zs))) Figure 24. Linear-time list reversal and its extracted recurrences.
show that it preserves identity, we also need the corresponding η/surjective pairing equation. However, these are true for the standard cartesian product of preorders. A reflection in Preorder × Preorder is a pair of reflections for each component. Unwinding these definitions gives the definitions of abs σ 0 ×σ 1 and conc σ 0 ×σ 1 in Figure 23.
The case of sums is more interesting. The standard coproduct of preorders X + Y is the disjoint union X ⊔ Y ordered as defined above. This extends to a 2-functor Preorder × Preorder → Preorder with f 0 + f 1 defined via case-analysis. This is bound-preserving because the branches of a case-analysis (on the standard coproduct in preorders) are a monotone position, and preserves identity/composition if we have βη equations for caseanalysis, which X + Y does.
For functions, the preorder of pointwise-ordered monotone maps X → Y extends to a mixed-variance 2-functor Preorder op × Preorder → Preorder, with functorial action given by pre-and post-composition. Moreover, a reflection abs ⊣ conc in Preorder is a reflection conc ⊣ abs in Preorder op with the roles of concretization and abstraction exchanged. This unpacks to the definitions of abs ρ→σ and conc ρ→σ in Figure 23, where abstraction precomposes with concretization, and vice versa.
Thus, while our general definition of model does not require types to be interpreted as 2-functors-for example, being a model does not require the η law for pairs that ensures preservation of identities-a number of more specific models will have this form, and thus admit the same definition of relativized model, given abstraction and concretization for base/inductive types. For example, we may freely apply in the interpretation of any type constructor, e.g. defining D σ 0 ×σ 1 to be O(D σ 0 × D σ 1 ) for more precision. 7.4.1. Example: list reverse. To get a sense of how polymorphic abstraction behaves, let us analyze the polymorphic linear-time list reverse function given in Figure 24 in the model W → V. We choose this model because on the one hand W provides enough information for analyzing monomorphic functions like sumtree that depend on more than just the usual notion of size, yet we still want to analyze a polymorphic function like list reversal in terms of list length, ignoring any information about the elements of the argument list. Since polymorphism in the source language arises only via let-bindings, the recurrence for rev ′ that is given is the recurrence that is substituted for for rev ′ according to the definition of extraction for let-expressions. A typical informal analysis of rev would really analyze rev ′ , and might define S(n, m) and T (n, m) to be the size and cost of rev ′ xs ys when xs and ys have length n and m, respectively. One would then observe that S and T satisfy the recurrences S(1, m) = m S(n, m) = S(n − 1, m + 1) T (1, m) = 1 T (n, m) = 1 + T (n − 1, m) from which one establishes the O(n) bound on cost.
Just as with our other models, to analyze rev, we must consider its instantiation at some arbitrary small type σ. In the model W, this would entail understanding how to compute Fold W s φ for arbitrary φ, which would be defined in terms of all φ ′ ≤ φ. The key point of W → V is that while we cannot avoid considering the instantiation of rev at arbitrary σ, we only need to know how to compute Fold W s φ for those φ that are the concretizations of values in V σ list . To see this, let us define φ σ list n = conc σ list (n)-observe that φ σ list n maps σ list to n and all other datatypes to ∞-and then compute rev ′ , where we write f σ φ for fold σ list xs of {nil ⇒ · · · | cons ⇒ · · ·} η{xs → φ}: ) When restricted to concretizations of abstract values, Fold W is straightforward to compute.
With this in mind, setS(n, m) = abs((f σ φ σ list n ) p φ σ list m ) p . Our goal is to write a recurrence forS(n, m). We start with (( (1, λzs.(0, zs) To computeS(n, m) for n > 1, we first compute Analysis of cost proceeds in a similar manner. We have again extracted the recurrences we expect from an informal analysis, but instead of those recurrences being in terms of arbitrary values in W σ list , they are in terms of the length of the argument list.
Stepping back a bit, recall from Section 7.1 that we can apply parametricity to the standard model to reason about the cost of rev xs, which seems comparable to what we have just done. But there is a difference. The result from parametricity tells us that the cost of the result is determined by the length of the argument, but it does not tell us how to compute the former in terms of the latter. What we have done here is to formally justify the recurrence that does just that. 7.5. Lower bounds and an application to map fusion. So far we have focused on extracting recurrences for upper bounds. However, the syntactic bounding theorem is agnostic with respect to the actual interpretation of the size order. We take advantage of this to derive recurrences for upper and lower bounds in the main constructor counting model of Section 7.2. Let us consider the map function given in Figure 25. By reasoning that is by map =λf ρ→σ , xs ρ list .
fold ρ list xs of nil ⇒ nil | cons ⇒ (x, r).cons(f x, r) map =λ(f : ρ → σ ), (xs : ρ list) now hopefully somewhat mundane, if we set T map f (n) = ( map f n) c , then we obtain the recurrence T map f (1) = 1 T map f (n) = 1 + (f ∞) c + T map f (n − 1). Solving this recurrence yields an upper bound of T map f (n) = n(1 + (f ∞) c ). Now let us apply this to the two sides of the usual map fusion law We hope to show that the right-hand side is less costly than the left. Working through the recurrence extractions, we conclude that the cost of the left-hand side is bounded by T map f •map g (n) = 2n(1 + (g ∞) c + (f ∞) c ), whereas the right-hand side is bounded by T map(f •g) (n) = n(1 + (g ∞) c + (f (g ∞) p ) c ). Even under the assumption that the costs of f and g are independent of their arguments does not result in the desired conclusion, because we only know that these recurrences yield upper bounds, and the fact that one upper bound is larger than another tells us nothing about the actual costs. What we would like to know is that these recurrences are tight, and for that we need lower bounds as well.
As we already mentioned, as long as we have a model of the recurrence language in which the interpretation of the size order satisfies the axioms of Figure 11, the bounding theorem holds. So to obtain lower bounds, we would want a model in which the order on the interpretation of C is the reverse of the usual order. That means we would have two models in hand, one that gives us upper bounds, and one that gives us lower bounds; we would then have to ensure that the recurrences in each model can be sensibly compared. As it turns out, we can arrange that by using the model in Section 7.2, because the interpretations of the types are all complete upper semi-lattices. We take advantage of the fact that a complete upper semi-lattice is in fact a complete lattice, where greatest lower bounds are defined by X = {x | ∀y ∈ X : x ≤ y}. This permits us to define the dual interpretation of the model (U sm Because all of the size-order axioms except (β δ ) and (β δfold ) are witnessed by identities in V (i.e., the left-and right-hand sides of the axioms have the same denotation), we can take the semantic functions in V * not related to datatypes to be those of V. For datatype-related functions, it is unnecessary to change either size F or C F ; the only change needed is that we define D * F (n) = {a | C F (a) ≥ n}. We can verify that (β δ ) holds by observing that and hence D * F (C F (a)) ≥ * a as required. Of course, the value of the destructor is different in this model, but not by much; a routine calculation shows that 1)); compare this to the calculation in Section 7.2.
We likewise can define the semantic fold function in this model by Returning to our discussion of comparing the costs of map f • map g and map(f • g) we now conclude that T ℓ map f • map g (n) = 2n(1 + (g ⊥) c + (f ⊥) c ) is a lower bound on the cost of map f • map g, so to show that map (f • g) is the more efficient alternative, it suffices to show that , which is trivial when the costs of f and g are independent of their arguments.

Recursion
We have not included general recursion in our languages in order to focus on the key idea that different models formally justify various informal cost analyses. The presence of recursion does not change this perspective, but it does complicate the model descriptions in ways orthogonal to our main thrust. We sketch the approach of Kavvos et al. [2020] here.
For the syntax, we add recursive definitions to the source language with a standard letrec construct and to the recurrence language with a standard fix constructor, corresponding to the usual approach for call-by-value and call-by-name languages. The details are given in Figure 26, where we also give two new size-order rules to replace (β δfold ). In these new rules, E is an elimination context and fix n x.e is defined by fix 0 x.e = fix x.x fix n+1 x.e = e{fix n x.e/x}.
The two rules codify the relation between the size order and the information order that is implicit in the presence of fix: a more defined bound is a better (i.e., smaller) bound. In the presence of non-termination, the bounding relation requires a slight adjustment: e E provided: if E terminates, then eθ ↓ n v, where n ≤ E c and v val E p . This is the only place a (standard) operational semantics is needed in the recurrence language, and we are investigating how to eliminate its use. For the semantics of the recurrence language, we impose additional structure on our applicative structures. We call the new structures sized domains and they are defined just like applicative structures, except that for each U ∈ U sm , is a preorder as before, and (D U , ⊑ U , ⊥ U ) is a complete partial order. The semantic domains must satisfy two additional constraints: • If x ⊑ U y, then y ≤ U x; and • If y 0 ⊑ U y 1 ⊑ U · · · and for all i, x ≤ U y i , then x ≤ y i . Source language: x.e ≤ σ fix n x.e Recurrence extraction: That leaves us with verifying that the models that we presented in Section 7 are sized domains. For each of the models, we take ⊑ N ∞ i to be the usual flat order with ⊥ N ∞ i = ∞ (again, cf. [Rosendahl, 1989]) extended pointwise and componentwise for functions and products. For sums, set X ⊑ Y if Y ⊆ X. It is a straightforward exercise to show that D ρ+σ is a CPO that satisfies the constraints just given. To show that we have a model, it suffices to verify that the semantic functions are simultaneously monotone with respect to ≤ and continuous with respect to ⊑, after which Prop. 7.1 can be extended with the clause that λ λa. Γ ⊢ e : σ η{x → a} is continuous with respect to ⊑. Verification of continuity for Case relies on two facts that hold in these models at all types: Extracting syntactic recurrences from general recursive functions and interpreting them in our models follows the same pattern we have already seen several times. But now the recurrences may have more complex solutions (such as poly-log solutions). For example, Kavvos et al. [2020] analyze the standard implementation of merge-sort and interpret it in the model of Section 7.2. Under the usual assumption that the cost of the comparison function is constant the recurrence clause of the semantic recurrence is T (n) = c + dn + T (n/2) for some constants c and d (that arise from the analyses of the functions that divide a list in two and merge two sorted lists), just as expected. Now one may reason in the semantics to establish the O(n lg n) cost from this recurrence.
Quick-sort provides an interesting example of how more complex models can be used to capture subtle information that may be necessary for an asymptotic analysis. Quicksort relies on a partitioning function part : α → α list → α list × α list such that part x xs = (ys, zs), where ys consists of the elements of xs that are < x and zs those elements that are ≥ x. A key part of the analysis of quick-sort is the fact that the sum of the lengths of ys and zs is the length of xs. In the models we have presented in Section 7, the extracted recurrence will not yield such a bound. For example, in the main constructorcounting model, the best we can conclude about the extracted recurrence is that in the semantics, part x n = (n, n). The problem is that the interpretation of products requires that we choose some specific pair that is a bound on all pairs (k, ℓ) such that k + ℓ = n, and (n, n) is the least such bound. But we have seen this situation before when it came to interpreting sums, and the solution is the same: instead of taking V ρ×σ = V ρ × V σ , we can instead take V ρ×σ = O(V ρ × V σ ). While the calculations become more tedious, in such a model we can show that part x n = {(k, ℓ) | k+ℓ ≤ n}. However, it turns out this is not quite enough. Both the source and recurrence languages have negative products, which means that projections must be used to extract ys and zs. In the interpretation of the extracted recurrence, projection of a set of pairs maximizes over the corresponding component, and so π i (part x n) = n (because n + 0 = 0 + n = n), which again leads to a weak bound. Instead, we must use positive products with an elimination of the form split (x, y) = e ρ×σ in e ′ . The corresponding elimination form in the recurrence language can be interpreted by maximizing e ′ over all pairs in e , which is precisely what is needed to carry out the rest of the usual analysis of quick-sort.

Related work
We first expand upon a couple of observations that we made earlier and mention some motivating history behind some technical details. Then we address how our work fits into the literature on cost analysis.
We touched on an application of parametricity in Section 7.1. Seidel and Voigtländer [2011] have interpreted free theorems [Wadler, 1989] to obtain relative complexity information. Their work can be viewed as applying parametricity to the standard model, but in a somewhat more general setting of a recurrence language that has a monadic type constructor C(σ) for "complexity of σ," with projections for cost and potential. They define a notion of lifting relations to complexities (much as relations are lifted to inductive types), which allows them to interpret a free theorem such as f (hd xs) = hd(map f xs) in such a way that the interpretations of both sides yield complexity information, and the identity then allows them to conclude, e.g., that the cost of the left-hand side is no greater than that of the right-hand side. With our approach, we would simply extract recurrences from the left-and right-hand sides and reason about them as in Section 7.5. While on the topic of relative cost information, we would be remiss to not mention the type-and-effect system of Ç içek et al. [2017], which permits a very precise analysis of the relative cost of different algorithms on the same arguments or the same algorithm on different arguments. We have not investigated whether our techniques can be adapted to provide comparable analyses.
We drew an analogy with abstract interpretation (AI) in Section 7.2.6 and made use of the existence of a Galois connection of the sort that arises in AI in Section 7.4. Rosendahl [1989] uses AI to extract cost bounds directly from a first-order fragment of Lisp. She first defines a program translation similar to our syntactic extraction and interprets it in the standard model D of S-expressions. She then defines an AI from P(D) into a finiteheight lattice of "partial structures," whose values are essentially truncated standard values. Given a notion of size s : D → N and a computable bound on λ λn.α({x | s(x) = n}), the interpretation of the syntactic recurrence in the abstract domain is a computable upper bound on the cost of the original program. This work is restricted to first-order programs and does not handle branching data structures well (e.g., if s(t) is the number of nodes in the tree t, then for n > 1, α({x | s(x) = n}) is a node structure that is truncated at its children, so the bounds are all trivial). But these ideas may provide an approach to computing bounds on semantic recurrences in models where the semantic recurrence itself is not computable (a situation that does not arise in the models we have presented).
While our notion of potential is drawn most directly from Danner and Royer [2007], it traces back at least to Shultis [1985], who defines a denotational semantics for a simple higher-order language that models both the value and the cost of an expression. He develops a system of "tolls," which play a role similar to that of our potentials. The tolls and the semantics are not used directly in calculations, but rather as components in a logic for reasoning about them. Sands [1990] defines a translation scheme in which each identifier f in the source language is associated to a cost closure that incorporates information about the value f takes on its arguments, the cost of applying f to arguments, and arity. Cost closures record information about the future cost of a partially-applied function, just as our potentials do. The idea of using denotational semantics to captures cost information has been seen before. We have already mentioned Rosendahl [1989] and Shultis [1985]. Van Stone [2003] defines a category-theoretic denotational semantics that uses "cost structures" (these include the C × − writer monads we use here) to capture cost information and shows that it is sound with respect to a cost-annotated operational semantics for a higher-order language. Our bounding theorem is roughly analogous to Van Stone's soundness theorem, but is a bit more general because we show an inequality (using the size order on the complexity language) instead of an equality, which allows the bounding theorem to apply to models with size abstraction.
Turning now to the literature on cost analysis, constructing resource bounds from source code has a long history in Programming Languages. The earliest work known to the authors is that of Cohen and Zuckerman [1974], which extracts programs that describe costs from an ALGOL60-like language that are intended to be manipulated in an interactive system, and Wegbreit's [1975] METRIC system, which extracts recurrences from simple first-order recursive Lisp programs. An interesting aspect of the latter system is that it is possible to describe probability distributions on the input domain (e.g., the probability that the head of an input list will be some specified value), and the generated bounds incorporate this information. Le Métayer's [1988] ACE system converts FP programs [Backus, 1978] (under a strict operational semantics) to FP programs (under a non-strict semantics) describing the number of recursive calls of the source program. The first phase is comparable to the cost projection of our recurrence extraction; the potential projection is the original program. Both METRIC and ACE yield non-recursive upper bounds on the generated cost functions (this is the bulk of the work for ACE). These systems are restricted in their datatypes and compute costs in terms of syntactic values; the notion of "size" is somewhat ad-hoc and second class. Many approaches to cost analysis rely on the idea that the cost can be treated as an additional output of the program, or as a piece of program state; Wadler [1992] observed that this can be represented by a monadic translation -though in our case we use the writer monad rather than the state monad, since we do not give programs access to their cost.
There are many approaches to type-based cost analysis [Crary and Weirich, 2000, Hofmann and Jost, 2003, Jost et al., 2010, Hoffmann and Hofmann, 2010, Hoffmann et al., 2012, Jost et al., 2017, Knoth et al., 2019, 2020, Avanzini and Dal Lago, 2017, Ç içek et al., 2017, Wang et al., 2017, Dal Lago and Gaboardi, 2011, Handley et al., 2019, Rajani et al., 2021. At a high level, these systems include special-purpose judgements or types that track cost, indexed or refinement types that track the size of values, and a type checking or inference mechanism that can automatically determine some resource bounds. For example, the Automatic Amortized Resource Analysis (AARA) technique of Hoffmann et al. [2012, Jost et al. [2017], Hofmann and Jost [2003], Jost et al. [2010], Hoffmann and Hofmann [2010], with an implementation at Hoffmann [2020], computes cost bounds by introducing a type system with size information that is parameterized by an integer degree, and then performing type inference. If inference is successful, then the program cost can be bounded by a polynomial of at most that degree (and a bound is reported); otherwise it cannot. As its name suggests, AARA automatically incorporates amortization, resulting in tighter bounds for some programs than our extracted recurrences yield (but see [Cutler et al., 2020] for an extension of our approach to amortized analysis). The basic AARA technique has been extended in numerous ways, e.g. with refinement types Knoth et al. [2019Knoth et al. [ , 2020 for synthesizing programs with desired resource bounds, and for more precise tracking of potential in values. The Timed ML system of Wang et al. [2017] also uses refinement types (indexed types in the style of DML [Xi and Pfenning, 1999]) that permit the user to define datatypes with their own notion of size and to include cost information in the program type. Type inference produces verification conditions which, if solvable, validate the cost information. That cost information may be very concrete, or left more open-ended, in which case the verification conditions end up synthesizing (recurrence) relations that must be satisfied. Avanzini and Dal Lago [2017] develop a non-amortized type-based analysis, which uses a translation similar to our recurrence extraction to explicitly represent the cost as a unary numeral. As a result, the evaluation cost of the original program is reflected in the size of the cost component of the translated program. They then make use of an extension of sized types [Hughes et al., 1996] to infer a type for the translated program, which therefore includes a bound on the cost in terms of the size of the arguments.
All of these type-based approaches are impressive in the breadth of successful analyses and/or automation thereof. However, we believe it is nonetheless worth studying cost analysis by recurrence extraction for several reasons. First, the process of inferring bounds using these specialized type systems and their associated solvers is not, in our opinion, very easy for a person to do, while our focus is on formalizing the method that we readily teach students to do. Second, automated approaches necessarily impose some limits on the kinds of bounds that can be inferred and the notions of size that are supported to facilitate inference (though Handley et al. [2019] also allows explicit proofs; see the discussion of techniques in proof assistants below). For example, AARA infers polynomial bounds, while our approach (adapted to the setting of general recursion) can produce recurrences with non-polynomial solutions. Third, type based approaches make the size and cost an intrinsic feature of the code: in approaches based on refinement types, one must, for example, define one tree type where size means number of nodes, and a different tree type where size means height, which causes code duplication if both are necessary; in amortized approaches, one must choose the potential annotations when defining a type (though sometimes this can be mitigated by parametrizing the datatypes Knoth et al. [2020]). In our approach, cost and size are an extrinisic property of the code, so the same function can be interpreted in different models with different notions of size for different analyses, which can be useful e.g. for a library function that is used in two different programs by other functions that require two different notions of size. That said, this does not address situations where two different notions of size for a type are needed in a single program -one possible solution is a model in which the potential is the pair of these sizes, but this would have similar reuse problems to changing a refinement type to include additional information, in that all existing analyses would formally need to be modified.
Let us now consider work that, like ours, externalizes cost from programs that are typed in a more-or-less standard type system. Avanzini et al. [2015] carefully defunctionalize higher-order programs to first-order programs in order to take advantage of existing techniques from first-order rewrite systems. This leverages existing technologies to great effect, but does not match the kind of recurrence extraction that we are aiming for in this work. The COSTA project [Albert et al., 2012] extracts cost recurrences from Java bytecode; Albert et al. [2013] provides techniques for constructing closed forms for both lower and upper bounds on these recurrences. This group has also pushed forward on parallel cost [Albert et al., 2018], something that Raymond [2016] has looked into in our setting, but the COSTA work has focused on first-order, low-level languages. Cutler et al. [2020] adapt our technique to handle amortized analysis. Reinforcing our goal of formalizing informal approaches, the source language there includes constructions for describing a credit allocation policy (the banker's method) and extraction of an amortized cost recurrence, to which a general theorem applies that total amortized cost bounds total actual cost. The language is sufficient for describing structures like splay trees in which the number of credits allocated to different parts of the structure is not constant, and the source language type system ensures that credits are not misused. The key point is that the amortized cost recurrence is extracted into essentially the same recurrence language as we have presented here, reflecting the fact that the recurrences that we use to describe amortized cost do not themselves refer to credits. Kavvos et al. [2020] give an approach to extending our technique to handle general (as opposed to structural) recursion by using call-by-push-value (CBPV) [Levy, 2003] as an intermediate source language into which both call-by-value and call-by-name can be embedded. While CBPV includes a fine stratification of types into computational and value types, analyzing a program still really just relies on notions of size and cost. Thus the syntactic recurrence language differs from the one just described only in replacing primitive recursion with a general fixpoint operator, along with corresponding axioms for the size order, thereby changing it from a version of System T with inductive types to a version of PCF with inductive types.
Atkey [2011], Guéneau et al. [2018], Charguéraud and Pottier [2019], Zhan and Haslbeck [2018] develop imperative program logics for reasoning about cost based on separation logic, essentially by treating the number of timesteps taken as part of the heap. A Coq or Isabelle implementation of these logics allows for reasoning about code, and the subgoals that arise during verification result in synthesizing recurrence relations, which play the role of our syntactic recurrences. While quite sophisticated algorithms and data structures can be analyzed this way, including imperative ones, for analyzing functional programs we find it more congruous to use (and teach to students) standard functional program verification techniques like inductive reasoning about outputs, as opposed to imperative program verification techniques like weakest precondition/characteristic formula generation. And as we note in Section 10, we conjecture that our approach extends to the analysis of many imperative programs, because the description of cost itself is frequently a functional description.
Turning now to semi-automated/manual reasoning in a functional style, Danielsson [2008] verifies a number of lazy functional programs in Agda using a dependent type tracking the number of steps a program takes. McCarthy et al. [2018] investigate a variant, implemented in Coq, using a monad parametrized by both the number of steps and a specification, given as a relation between the cost and value. The specifications are used both for functional correctness and for reasoning about cost, and this design allows Coq's extraction to OCaml to erase all costs and reasoning about them. The library also provides a source-to-source translation that translates simply-typed code into the monad, inserting appropriate ticks, which is analogous to our recurrence extraction. Radiček et al. [2017] define a specification logic for reasoning about monadic costs as an extension of higher-order logic. Benzinger's [2004] ACA system might be the closest in philosophy to ours, in that it extracts (higher-order) recurrences from call-by-name NuPrl programs that bound the cost of those programs. There we find (moderately complex) expressions that correspond to applying higher-order functions to arguments (necessarily alternating with projections) to describe the cost of a fully applied function argument, corresponding to our notion of higher-order potential. But this does not address more realistic call-by-value or call-by-need evaluation.
Since these approaches [Danielsson, 2008, McCarthy et al., 2018, Radiček et al., 2017, Benzinger, 2004 take place inside of a general-purpose logic or proof assistant, one can express costs in terms of the sizes of inputs by explicitly referring to an appropriate size function and proving how operations transform the size. Relative to this, a main contribution of our approach is to systematize and partially automate the reasoning about size, in the sense that our semantic interpretation of the potential of a function f gives a direct inductive definition of the fused "size of the result of f on inputs of size -" function. This is possible because we step outside of the programming language into a denotational setting where e.g. arbitrary maximums exist. We claim that this corresponds better to informal analyses than using the full power of a proof assistant to carefully prove how functions act on sizes, because the fused size-to-size function will simplify in ways that the original function does not. For example, because in these models most or all contexts are monotone in the size order, one can freely ignore branches whose size is dominated by another.

Conclusions and further work
We have presented a technique for extracting cost-and-size recurrences from higher-order functional programs that provably bound the operational cost in terms of user-definable notions of size, thereby giving a formal account of the process of many informal cost analyses. The technique applies to the pure fragment of strict languages such as ML and OCaml. Although we have not investigated the question carefully, it also seems that it applies to much reasoning about imperative programs. The reason is that such analysis often consists of extracting functional cost recurrences whose validity only depends on the fact that certain imperative operations have certain costs. For example, the analysis of many functions on an arrays depends on the fact that indexed access and update is constant time. But the analysis does not typically result in a recurrence that even refers to an array, much less destructively updates one. In our setting, we would either hard-code the costs of access and update in the syntactic recurrence extraction, or we would leave those functions as identifiers and analyze the semantic recurrence under the assumption that those identifiers are interpreted by constant-time functions. The de facto standard for such reasoning is Separation Logic, and the work that ours seems closest to in spirit is that of Zhan and Haslbeck [2018]. Our goal would be to provide relatively simple approaches to formalizing reasoning about many imperative programs. This is certainly speculative, and we have not investigated how far one can push this idea before requiring the machinery of something comparable to Separation Logic.
A natural direction to extend our work would be to handle cost analysis of lazy languages. Okasaki [1998] describes a technique of amortized analysis in which costs are split into "shared" and "unshared" costs in order to correctly account for the memoization of computations, and we believe our approach can be adapted to formalize this technique. Hackett and Hutton [2019] show that lazy evaluation is a form of "clairvoyant" call-byvalue and that cost can be described non-deterministically rather than in terms of shared and unshared costs. We hope to adapt our approach to yield corresponding recurrences, especially as they actually compute costs via an interpretation in a denotational model that appears to mesh nicely with our approach.
We have presented several models making use of different notions of size. It is no surprise that it is easier to work in models with simpler notions of size, and we saw in Section 7.4 that a simpler notion of size corresponds to a more abstract model. Formalizing the connection between more abstract and more concrete models so that information from the latter may be pulled into the former, would improve the usefulness of this sort of reasoning. This sounds like an analogy with safety and liveness theorems from abstract interpretation, and this is probably a fruitful direction for further study. More complex models should enable more sophisticated analysis. For example, the average case complexity of deterministic quick-sort can be described by assuming a (uniform) probability distribution on the inputs. That would seem to correspond to interpreting the usually extracted recurrence in a model in which inductive types are interpreted by probability distributions or random variables. Barnaby [2018] has made preliminary progress in this direction, which indicates that it is probably necessary to have at least limited forms of dependent typing in the recurrence language.
We have focused on the extraction of semantic recurrences to show that they are the ones that are expected from informal analysis. We have not studied techniques for solving the semantic recurrences, which in general are higher-order functions. Benzinger [2004] discusses techniques for solving them by reducing them to first-order recurrence equations and then using off-the-shelf solvers such as Mathematica and OCRS [Kincaid et al., 2017]. Another fruitful direction would be to formalize the extracted semantic recurrences in proof assistants and make use of the formalization of standard theorems like the Master Theorem and of asymptotic reasoning as in Guéneau et al. [2018]. This would permit a formal development in a setting where complete automation is not possible.
The extraction of the syntactic recurrence is straightforward to implement, and a future project is to produce an end-to-end tool from source code to semantic recurrence to solution. We know that automated cost analysis is a complex project that many have attempted, and so this goal as stated is probably too ambitious, and we warn the reader that our thoughts here are pies in the sky at the time of writing. Our vision is more along the lines of an interactive system, in which recurrences are extracted and "easy" ones solved, but allowing the user to step in to provide assertions (hopefully proved!) about the solutions to difficult ones. Familiarity with recurrence extraction as a cost analysis technique would hopefully lower the entry barrier of such a tool. We could also hope that that same familiarity would enable users to work backward from an unexpectedly poor recurrence to the code from which it results (cf. Benzinger [2004]).  adapt AARA to provide worst-case inputs that validate the tightness of the produced bounds, which could be used to similar effect. Another direction such a project could take would be to pull either the syntactic or the semantic information back as additional interface-level components of a language library, so as to modularize cost reasoning and take advantage of the compositionality of our approach. However, this is not so straightforward. One issue that arises is that the denotation a type that is appropriate for analyzing an algorithm is not necessarily the one that is appropriate for using it. For example, the recurrence extraction approach works best to analyze binary search tree algorithms in terms of their heights, but a client who uses a binary search tree implementation is probably more interested in understanding the cost in terms of the size. This is a setting in which composing recurrences does not work as smoothly as we might hope. Understanding how to mesh them together, and more generally how to hide analyses that possibly require more complex types (such as those by Cutler et al. [2020]) behind an interface, is ongoing work.
Appendix A. Type preservation for the source language Type preservation depends on the usual substitution lemmas.
We now have the type preservation theorem.
Proof. The proof is a simultaneous induction on the height of the derivation that referred to in each part. We give just a few of the more interesting cases, starting with part (1). Case: xθ ↓ θ(x). By the hypothesis, ⊢ xθ : σ, so by the typing rules for closures, there must be some Γ ′ such that Γ ′ (x) = ∀ α.ρ and σ = ρ{ σ/ α}, and θ is a Γ ′ -environment. But that means that in particular, ⊢ θ(x) : ρ{ σ/ α}, as required.
Case: (fold δ e ′ of x.e)θ ↓ v. The typing must have the form θ a Γ-environment ⊢ (fold δ e ′ of x.e)θ : σ and the evaluation must have the form where without loss of generality we assume y / ∈ dom Γ and y / ∈ dom θ.
Case: F = F 0 × F 1 . Assumption (2) and inversion tells us that v = (v 0 , v 1 ), and assumption (4) tells us that We must show that (v ′′ 0 , v ′′ 1 ) val ( F 0 [(y : ρ ).E ′ , π 0 E], F 1 [(y : ρ ).E ′ , π 1 E]), for which it suffices to show that v ′′ i val F i [(y : ρ ).E ′ , π i E] for i = 0, 1. To do so we apply the induction hypothesis taking F i for F , v i for v, π i E for E, and v ′′ i for v ′′ . Verifying the assumptions is straightforward, noting that (3) follows because the derivation that as required. The assumptions for the induction hypothesis are straightforward to verify, noting that (3) follows because the derivation that v i val E i is a subderivation of ι i v i val E.
Case: Γ ⊢ c δ e : δ. Fix θ val Θ. We must show that (c δ e)θ (c, c δ p){Θ}, where e = (c, p). The evaluation of (c δ e)θ has the form eθ ↓ n v (c δ e)θ ↓ n c δ v Cost: n ≤ c{Θ} by the IH. Value: By the IH we have that v val F [δ] p{Θ}, and so by Lemma 5.1, v val F,δ p{Θ}. Since c δ p ≤ c δ p, the value bound follows by definition of val δ .