3.1.1 Number of generated tests

The program $\mathcal{P}$ shown in listings 1 and 2 gives rise to a probabilistic number of test cases. We study the questions: Is this number finite? If so, what is the expected number of test case?

Fig. 1. The Markov chain corresponding to $\mathcal{P}$ with $\mathsf{Init}=\sharp$ and blocks $\alpha \in \{1,\ldots ,r\}^*$ surrounded by blue boxes.

The program $\mathcal{P}$ gives rise to an infinite Markov chain, which is based on the SLD-tree corresponding to $\mathcal{P}$ . Recall that $\mathcal{P}$ is governed by some probabilities p_cont, p_1, …, which we will denote by $p_c$ , $p_1,\ldots , p_r$ . The Markov chain is depicted in Figure 1. Note that we model choice points via the states $s_i$ . This is necessary, because $\mathcal{P}$ will backtrack when a call to command1 fails, and proceed to command2 with probability $p_2$ . Double-circles denote output states – that is, whenever such a state is visited, a test case terminating in that state is generated. Node $\sharp$ corresponds to the empty list. $\bot$ is the only absorbing state. It corresponds a termination of the resolution algorithm.

The blue boxes denote areas that share a common structure. We call these areas blocks. We can uniquely identify each block by a finite sequence $\alpha \in \{1,\ldots ,r\}^*$ . For any state $s$ in the Markov chain, we denote by $\mathsf{Block}(x)$ the unique block that contains it. For any label occurring in a block ( $s_1,s_2,\ldots ,s_r$ and $c_1,\ldots ,c_r$ ) and a block $\alpha$ , write $s_1^\alpha$ , $c_1^\alpha$ , …for the unique state with that label in block $\alpha$ . In this way, we can identify any state in the Markov chain. Put differently, the Markov chain is given by a state space $\mathcal{S}=\{s_i^\alpha ,c_i^\alpha \mid 1 \leq i \leq r, \; \alpha \in \{1,\ldots ,r\}^*\}\cup \{\bot ,\sharp \}$ and transition probabilities $p(s,s')$ for $s,s'\in \mathcal{S}$ :

\begin{align*} p(\sharp ,s_1^{\varepsilon })&=p_c\\[4pt] p(\sharp ,\bot )&=1-p_c\\[4pt] p(s_i^\alpha ,s_{i+1}^\alpha )&=1-p_i && 1\leq i \lt r\\[4pt] p(c_i^\alpha ,s_{i+1}^{\alpha })&=1-p_c && 1\leq i \lt r\\[4pt] p(c_i^\alpha ,s_{1}^{\alpha \cdot i})&=p_c && 1\leq i \leq r\\[4pt] p(s_i^\alpha ,c_{i}^\alpha )&=p_i && 1\leq i \leq r \end{align*}

The dashed upward arrows (which correspond to backtracking to a lower-recursion level) are somewhat more technical to define. Those arrows originate in states of the form $s_r^\alpha$ or $c_r^\alpha$ . There are several cases to consider:

1. a) $\alpha \in \{1,\ldots ,r\}^*\cdot i$ for some $1\leq i\lt r$

2. b) $\alpha \in \{1,\ldots , r\}^*\cdot i \cdot r^+$ for some $1\leq i \lt r$

3. c) $\alpha \in r^*$

This motivates the following transition probabilities

\begin{align*} p(s_r^{\alpha \cdot i},s_{i+1}^\alpha )&=1-p_r && 1\leq i \lt r & \text{case a)}\\[4pt] p(c_r^{\alpha \cdot i},s_{i+1}^\alpha )&=1-p_c && 1\leq i \lt r & \text{case a)}\\[4pt] p(s_r^{\alpha '\cdot i\cdot r\cdots r},s_{i+1}^{\alpha '})&=1-p_r && 1\leq i\lt r & \text{case b)}\\[4pt] p(c_r^{\alpha '\cdot i\cdot r\cdots r},s_{i+1}^{\alpha '})&=1-p_c && 1\leq i\lt r & \text{case b)}\\[4pt] p(s_r^{r\cdots r},\bot )&=1-p_r && & \text{case c)}\\[4pt] p(c_r^{r\cdots r},\bot )&=1-p_c && & \text{case c)} \end{align*}

We call edges from a block $\beta \cdot \alpha$ to a state in block $\beta$ or from any block to $\bot$ an upward edge. They correspond precisely to the dashed arrows in Figure 1. If the Markov chain follows such an edge, we say block $\beta \cdot \alpha$ is left upward or that the chain traverses upward at that point. A block that has been left upward, is never visited again.

It is immediate that every state is visited at most once. There are no two states that can be reached from one-another. Note further that if we omit the dashed arrows and the state $\bot$ , the resulting graph structure is an infinite, finitely branching tree. Yet, it is conceivable that the terminal state $\bot$ is never reached, because the sequence of states visited from $\sharp$ is infinite. The following proposition shows that this is not the case, provided $p_c\lt 1$ .

Corollary 1. Let $\alpha$ be any block. The probability of reaching $\alpha$ from $s_1^\varepsilon$ (i.e. from the initial block) is:

\begin{align*} \Pr [H^{\alpha }\lt \infty ] = p_c^{|\alpha |}\cdot \prod _{i=1}^{|\alpha |}p_{\alpha _i}\lt \infty \end{align*}

Consequently, the probability of reaching $\alpha$ from $\sharp$ is $p_c^{|\alpha |+1}\cdot \prod _{i=1}^{|\alpha |}p_{\alpha _i}$ .

Let $s\in \mathcal{S}$ . We denote by $N(s)$ the random variable that counts the total number of states visited from $s$ (including those in downstream blocks), before $\mathsf{Block}(s)$ is left upward. A useful observation is that $N(s)=H^{E}_{s}$ can also be expressed as a hitting time, where $E=\{s_i^\beta \mid \beta \prec \mathsf{Block}(s), \; 1\leq i\leq r\}\cup \{\bot \}$ . Note that it would suffice to take the subset of $E$ which contains $s_{\alpha _i+1}^\beta$ for any $\beta =\alpha _1\cdots \alpha _{i-1} \prec \alpha$ . To define this set, we would have to work around the case $\alpha _i=r$ – indeed, if $\alpha \in r^*$ , then $E=\{\bot \}$ . So we define $E$ as larger than needed purely to simplify notation. Note moreover that $E$ depends on $\alpha =\mathsf{Block}(s)$ . Since $\alpha$ is usually clear from context, we simply write $E$ , but also use the notation $E_\alpha$ when needed.

Proof. Let $\alpha =\mathsf{Block}(s)$ . It is obvious that $N(x^\alpha )\leq N(s_1^\alpha )$ for all $x\in \mathcal{S}$ with $\mathsf{Block}(x)=\alpha$ . It therefore suffices to show that $N(s_1^\alpha )$ is finite. In the remainder of this proof, we write $\hat {s}=s_1^\alpha$ .

Let now $\beta$ be any block and let $M_\beta$ denote the number of states visited in block $\beta$ from $s_1^\beta$ . Clearly $M_\beta \leq 2r$ . Let furthermore $I_\beta =\unicode {x1D7D9}_{H^{\beta }_{\hat {s}}\lt \infty }$ denote the indicator random-variable of the event that $\beta$ is visited from $\hat {s}$ . Note that both random-variables are independent because the underlying random events in $\mathcal{P}$ are independent and we count by $M_\beta$ only states that are visited once $\beta$ is entered. We have:

\begin{align*} N(\hat {s})=\sum _{\beta \in \alpha \cdot \{1,\ldots ,r\}^*}I_\beta \cdot M_\beta \end{align*}

There are precisely $r^l$ blocks that have distance $l\in \mathbb{N}$ from $\alpha$ . For each such block $\beta =\alpha \cdot \beta _1\cdots \beta _l$ , the probability of reaching it from $\alpha$ is $\Pr [I_\beta =1]=p_c^{l}\cdot \prod _{i=1}^lp_{\beta _i}$ by Corollary 1 (if $l=0$ then $\beta =\alpha$ and the probability is 1). Then $\Pr [I_\beta =1]\leq (p_{\mathsf{max}}\cdot p_c)^l$ for all $\beta$ . This gives (using linearity of expectation and that $I_\beta$ is independent from $M_\beta$ for all $\beta$ ):

\begin{align*} \mathsf{E}[N(\hat {s})]&=\sum _{\beta \in \alpha \cdot \{1,\ldots ,r\}^*}\mathsf{E}[I_\beta ]\cdot \mathsf{E}[M_\beta ] \leq \sum _{l=0}^\infty r^l\cdot \big(p_c^l\cdot p_{\mathsf{max}}^l\big) \cdot 2r\\ &= 2r\cdot \sum _{l=0}^\infty \eta ^l = \frac {2r}{1-\eta }\\[-42pt]\nonumber \end{align*}

Let $s\in \mathcal{S}$ . Denote by $O(s)$ the number of output states that are visited from $s$ before $\mathsf{Block}(s)$ is left upward. Clearly $O(s)\leq N(s)$ .

Theorem 1. Let $p_c\lt 1$ and $p_c\cdot r\cdot \max \{p_1,\ldots , p_r\}\lt 1$ . Then for any block $\alpha$

\begin{align*} \mathsf{E}[O(s_1^\alpha )]=\frac {\sum p_i}{1-p_c\sum p_i}\quad \text{ and } \quad \mathsf{E}[N(s_1^\alpha )]=\frac {r + \sum p_i}{1-p_c\sum p_i} \end{align*}

Proof. $C\stackrel {\text{ def}}{=}\mathsf{E}[N(s_1^\alpha )]$ is finite by Lemma1. Note that $C$ is independent of $\alpha$ by the strong Markov property. We recall that $N(s_1^\alpha )=H^{A}_{s_1^\alpha }$ is a hitting time, where $A=\{s_i^\beta \mid \beta \prec \alpha \}\cup \{\bot \}$ . In the remainder of the proof, we drop the superscript Greek letter for all states in $\alpha$ ; that is, $s_1$ is understood to mean $s_1^\alpha$ .

Every path from $s_1$ to $A$ must visit $s_2,\ldots , s_r$ . Thus, by the strong Markov property, $N(s_1) = \left ({\sum _{i=1}^{r-1}} {H^{s_{i+1}}_{s_i} }\right )+ {H^{A}_{s_r}}$ . By linearity of expectation and Eq. (1)

(*) \begin{align} C =\mathsf{E}[N(s_1)] &= \sum _{i=1}^{r-1}h^{s_{i+1}}_{s_i} + h^{A}_{s_r} = \sum _{i=1}^{r-1} 1 + p_i\cdot h^{s_{i+1}}_{c_i} + (1+p_r\cdot h^{A}_{c_r})\nonumber\\ & = \sum _{i=1}^{r-1} 1 + p_i\left(1 + p_c\cdot \underbrace {h^{s_{i+1}}_{s_1^{\alpha \cdot i}}}_{C}\right) + \left(1+p_r\left(1 + p_c \cdot \underbrace {h^{A}_{s_1^{\alpha \cdot r}}}_{C}\right)\right)\nonumber\\ & = r + \sum _{i=1}^{r} p_i + Cp_c\sum _{i=1}^{r} p_i \end{align}

Solving for $C$ proves the second claim of the theorem.

The proof for $\mathsf{E}[O(s_1)]$ is similar. We first make a slight modification to Eq. (1) to count only output states:

\begin{align*} \begin{array}{ll} h^{A}_{s}=0 &\quad \text{if $s\in A$}\\[3pt] h^{A}_{s}=\begin{cases} 1 + \sum _{s'\in \mathcal{S}} p(s,s')h^{A}_{s'} & s\text{ is an output state}\\[3pt] 0 + \sum _{s'\in \mathcal{S}} p(s,s')h^{A}_{s'} & s\text{ is not an output state} \end{cases}&\quad \text{if $s\notin A$} \end{array} \end{align*}

This can be shown in exactly the same way as equations (1) (see e.g. Norris (Reference Norris1998) for the proof of the classical theorem; the adaption is straightforward). Alternatively, the following intuition can be turned into a formal proof:

Observe that in counting only output states $\mathcal{O}\subseteq \mathcal{S}$ , we are effectively studying a second Markov chain, whose state set consists only of output states (and $\bot$ ). For $s, s' \in \mathcal{S}$ write $\mathsf{P}(s,s')$ for the set of all simple paths (without repeating vertices) from $s$ to $s'$ that do not visit $\mathcal{O}$ . The transition probabilities $p'$ of this second chain are given by the relation

\begin{align*} p'(s,s') = \sum _{w\in \mathsf{P}(s,s')} \prod _{i=1}^{|w|-1} p(w_{i-1},w_i) \end{align*}

So our modified formulas are simply a different way to write Eq. (1) for this modified chain in an iterative fashion.

With these modifications, we see that $(\ast )$ becomes:

\begin{align*} C' = \sum _{i=1}^{r} p_i + C'p_c\sum _{i=1}^r p_i \end{align*}

Again, solving for $C'$ establishes the claim.

Note that for any given state $s_1^\alpha$ , the mean hitting time $h^{s_1^\alpha }_{\sharp }=\sum _{n\geq 1}\Pr [H^{s_1^\alpha }_{\sharp }\geq n]\geq \sum _{n\geq 1} 1-p_c=\infty$ (where we use $\mathsf{E}[X]=\sum _{n\geq 1} \Pr [X\geq n]$ for any random variable that only takes on positive integer values). So although we have a non-zero probability of selecting every test, we won’t, informally speaking, do so on average. Naturally, this is solved by repeating the experiment a sufficient number of times. This is the content of the next section.

3.1.2 Infinite looping and time-to-hit

As shown in Lemma1, the program in listing 1 terminates eventually. As a result, every state except $\bot$ in the Markov chain we studied above is transient and, moreover, the number of produced test cases is always finite. In testing, one aims at a high test coverage, and the number of test cases we produce in this fashion, though free of duplicates, has a low chance of visiting tests in deep blocks. A natural approach is to loop on the predicate t/1 like so:

With respect to our Markov chain this amounts to removing $\bot$ and to instead redirect any arc into $\bot$ to $\sharp$ . The resulting chain is recurrent (indeed positive recurrent) and we compute the mean hitting time of any state. In what follows, we will assume $p_1=p_2=\cdots =p_r\stackrel {\text{ def}}{=} p$ such that $r\cdot p \cdot p_c\lt 1$ (as in Theorem1). Moreover, we assume that $p(\sharp ,s_1^{\varepsilon })=1$ , so that the empty list is never selected as an output. This simplifies the formulas below slightly, but has otherwise no effect on the line of reasoning we give here.

Given the conditions of Theorem1, there is a constant $C=N(s_1^\alpha )$ that is independent of the value of $\alpha$ . As noted before, $C=h^{E_\alpha }_{s_1^{\alpha }}$ is a mean hitting time where $E_\alpha =\{s_i^\beta \mid \beta \prec \alpha ,\;1\leq i\leq r\}\cup \{\sharp \}$ (note that we modified the definition of $E$ used in the previous section by replacing $\bot$ by $\sharp$ ). Recall that we usually drop the subscript $\alpha$ , because it is clear from context.

If we hop from one state $s_i^\alpha$ to its neighbor $s_{i+1}^\alpha$ we might traverse the tree below $s_{1}^{\alpha \cdot i}$ with probability $p\cdot p_c$ . That step will visit $C$ states. This means (by Eq. (1)):

\begin{align*} h^{s_{i+1}^\alpha }_{s_i^\alpha } = 1 + p(1 + p_cC) \stackrel {\text{ def}}{=} \Delta \end{align*}

More generally the mean hitting time within a block is again independent of $\alpha$ and can be computed as:

(2) \begin{equation} h^{s_i^{\alpha }}_{s_1^{\alpha }} = h^{s_2^{\alpha }}_{s_1^{\alpha }} + h^{s_3^{\alpha }}_{s_2^{\alpha }}+\cdots + h^{s_i^{\alpha }}_{s_{i-1}^\alpha }= (i-1)\Delta \end{equation}

We define the leave upward time $U_{s_i^\alpha }=h^{E}_{s_i^\alpha }$ where $E=E_\alpha$ as above. Note that the value $U_{s_i^\alpha }\in \mathbb{N}\cup \{\infty \}$ does not actually depend on $\alpha$ . This justifies writing $U_i=U_{s_i^\alpha }$ . It is obvious that $C=U_{1}$ . Moreover, by using the same derivation as that in Eq. (2):

(3) \begin{equation} U_i = (r-i + 1)\Delta \quad 1\leq i \leq r \end{equation}

We already noted that $C=U_1$ . A related quantity is the hitting time of $\sharp$ from any $s_i^\alpha$ , $\alpha =\alpha _1\cdots \alpha _t$ , which we may compute using the intermediate leave upward times:

\begin{align*} h^{\sharp }_{s_i^\alpha } = U_{i} + U_{\alpha _t + 1}+ \cdots U_{\alpha _1+1} \end{align*}

Note that we abuse notation: Equation (3) gives $U_{r+1}=0$ . While $s_{r+1}^\alpha$ does not exist and hence the corresponding hitting time is not defined, it is convenient to allow such terms and exploit that $U_{\alpha _j+1} = 0$ whenever $\alpha _j=r$ ( $1\leq j\leq t$ ).

With this, we may compute:

(4) \begin{align} h^{\sharp }_{s_i^\alpha } &= U_i + \sum _{k=1}^t U_{\alpha _k+1} = \Delta \cdot \left ((r-i+1) + \sum _{k=1}^t (r-(\alpha _k+1) + 1) \right )\nonumber \\ &= \Delta \cdot \left (1+ \sum _{k=0}^t r- \alpha _k\right )\qquad \text{where }\alpha _0\stackrel {\text{ def}}{=} i \end{align}

Note again that the formula works correctly, if $i=r+1$ : Say $\alpha = rrr$ . Then we are in the process of falling back to $\sharp$ and the equation gives 0. While the hitting time is again not defined for the non-existent state $s_{r+1}$ , we will sometimes have to compute the hitting time of $\sharp$ from the “right neighbor” of $s_{i+1}$ . In these situations, abusing notation in this way is useful because we need not distinguish between cases where $i \lt r$ and those where  $i=r$ .

Finally, we may now compute the hitting time of an arbitrary state in terms of hitting times in intermediate blocks, again using Eq. (1). Let $\alpha =\beta \cdot j$ .

\begin{align*} && h^{s_1^\alpha }_{\sharp } & = h^{s_j^\beta }_{\sharp } + 1\\ &&&\quad +(1-p)\left(h^{\sharp }_{s_{j+1}^\beta } + h^{s_i^\alpha }_{\sharp}\right) & \big(\text{fall through to $s_{j+1}^\beta $}\big)\\ &&&\quad + p\left(1 + (1-p_c)\left(h^{\sharp }_{s_{j+1}^\beta } + h^{s_i^\alpha }_{\sharp }\right)\right) & \left(\text{no visit to next block $\alpha $}\right)\\ && &= h^{s_j^\beta }_{\sharp } + 1 + p+ \left(h^{\sharp }_{{s_{j+1}^\beta }}+h^{s_i^\alpha }_{\sharp }\right)(1 - pp_c) \end{align*}

This gives

\begin{align*} h^{s_1^\alpha }_{\sharp } = \frac {h^{s_j^\beta }_{\sharp } + 1 + p + (1-pp_c)h^{\sharp }_{{s_{j+1}^\beta }}}{pp_c} \end{align*}

and together with Eq. (2) and Eq. (4), recalling that $\alpha _{|\alpha |}=j$ , we have:

(5) \begin{align} h^{s_i^\alpha }_{\sharp } &= \frac {h^{s_j^\beta }_{\sharp } + 1 + p + (1-pp_c)h^{\sharp }_{{s_{j+1}^\beta }}}{pp_c} + (i-1)\Delta \nonumber \\ &= \frac {h^{s_j^\beta }_{\sharp }}{pp_c}+\frac {1}{pp_c}\left (1 + p + (1-pp_c)\left(\sum _{k=1}^{|\alpha |}r-\alpha _k\right)\Delta \right ) + (i-1)\Delta \end{align}

The following theorem gives a closed formula:

Theorem 2. Let $s_i^\alpha$ for some $\alpha =\alpha _1\cdots \alpha _t$ . Let $\nu =pp_c$ and $\nu \cdot r\lt 1$ . Then

(6) \begin{equation} h^{s_i^\alpha }_{\sharp } = \nu ^{-t} + (i-1)\Delta + \sum _{k=1}^{t} \frac {1+ p + (\alpha _k-1)\Delta + (1-\nu )\sum _{s=1}^k(r-\alpha _s)\Delta }{\nu ^{t+1-k}} \end{equation}

Proof. By induction on $t$ . If $t=0$ , then $\alpha =\varepsilon$ and by Eq. (2), we have $h^{s_i^\varepsilon }_{\sharp }=1 + (i-1)\Delta$ . Moreover the empty sum in Eq. (6) equates to 0 establishing the induction base.

Now let $t\gt 0$ and assume the statement holds for $t-1$ . By induction, we may replace $h^{s_j^\beta }_{\sharp }$ in Eq. (5) with Eq. (6):

\begin{align*} &\frac {1}{\nu }\left (\nu ^{-(t-1)}+ (\alpha _t-1)\Delta + \sum _{k=1}^{t-1} \frac {1+p+(\alpha _k-1)\Delta +(1-\nu )\sum _{s=1}^k(r-\alpha _s)\Delta )}{\nu ^{t-k}}\right )\\ &\qquad + \frac {1}{\nu }\left(1 + p + (1-\nu )\left(\sum _{s=1}^{t}r-\alpha _s\right)\Delta \right ) + (i-1)\Delta \\ &\quad = \nu ^{-t}+ \sum _{k=1}^{t-1} \frac {1 + p + (\alpha _k-1)\Delta +(1-\nu )\sum _{s=1}^k(r-\alpha _s)\Delta )}{\nu ^{t+1-k}}\\ &\qquad + \frac {(1 + p + (\alpha _t-1)\Delta + (1-\nu )(\sum _{s=1}^{t}r-\alpha _s)\Delta )}{\nu ^{t+1 - t}} + (i-1)\Delta \\ &\quad = \nu ^{-t} + (i-1)\Delta + \sum _{k=1}^{t} \frac {1+ p + (\alpha _k-1)\Delta + (1-\nu )\sum _{s=1}^k(r-\alpha _s)\Delta }{\nu ^{t+1-k}}\\[-42pt]\nonumber\end{align*}

Proof. Write Eq. (6) as

\begin{align*} \nu ^{-t} + A + \nu ^{-t-1}\sum _{k=1}^t \frac {B_k +\sum _{s=1}^kD_{s}}{\nu ^{-k}} \end{align*}

for suitable constants (in $t$ ) $A\geq 0$ , $B_k\geq 0$ , and $D_{s}\geq 0$ , whereby $h^{s_i^\alpha }_{\sharp }\in \Omega (\nu ^{-t})$ .

Choose suitable largest values $B\geq B_k$ for all $t\in \mathbb{N}$ , $1\leq k\leq t$ , and $D\geq D_{s}$ for all $1\leq s\leq t$ . Bound Eq. (6) from above by

\begin{align*} \nu ^{-t} + A + \nu ^{-t-1}\sum _{k=1}^t \frac {B + D\cdot k}{\nu ^{-k}} \leq \nu ^{-t} + A + \nu ^{-t-1}(B+D)\cdot \sum _{k=1}^{t} \frac {k}{\nu ^{-k}} \end{align*}

A well-known calculation via derivatives gives $\sum _{k=1}^{t}k\cdot \nu ^k=\nu \sum _{k=1}^{t}k\nu ^{k-1} \leq \nu \sum _{k=1}^{\infty} $ $k\nu ^{k-1} = \nu \cdot \frac {\mathrm{d}}{\mathrm{d}\nu }\sum _{k=0}^{\infty }\nu ^k =\frac {\nu }{(1-\nu )^2}$ . With that we have

\begin{align*} \nu ^{-t} + A + \nu ^{-t-1}\sum _{k=1}^t \frac {B + D\cdot k}{\nu ^{-k}}\leq \nu ^{-t}+A+(B+D)\frac {\nu ^{-t}}{(1-\nu )^2}\in {\mathcal{O}}(\nu ^{-t})\\[-42pt] \nonumber\end{align*}

3.2.1 Number of generated tests

To study the number of generated tests in this context, we again have to define a Markov chain. The Markov chain in the shuffle and drop scenario is significantly more complicated than the one we studied previously in subsection 3.1.1. This is because there are now multiple ways a given test case can be output. To distinguish between those, we need to use a larger and more complex state set. We will illustrate this, before defining the Markov chain formally.

Consider again the program in listing 1. When the current goal is command(H), we have a set $R$ of rules whose head unifies with this goal. In this case, $R=\{\texttt{command(H) :-} $ $\texttt{command1(H)},\ldots ,\texttt{command(H) :- commandr(H)}\}$ . Recall this is meant to represent $r\in \mathbb{N}$ distinct rules. In standard SLD-resolution, we would select the first rule that occurs in the input program $\mathcal{P}$ , namely command(H) :- command1(H) first, and push the remaining $r-1$ rules on the stack from right to left. During backtracking, we would then eventually explore each of those rules in the order given in the input program (except in case of an infinite recursion).

In the drop-and-shuffle strategy, we first perform an independent Bernoulli trial for each rule $\rho \in R$ : With probability $p_d$ , we remove $\rho$ from $R$ . We refer to $p_d$ as the drop probability. In this way, a set $R'\subseteq R$ is computed. The random variable $|R'|$ follows a Binomial distribution: $\Pr [|R'|=k] = \binom {|R|}{k}p_d^{|R|-k}(1-p_d)^k$ . Next, we shuffle the set $R'$ . To this end, we select a permutation $\pi \in \mathbb{S}(R')$ , where $\mathbb{S}(M)$ denotes the symmetric group on a given set $M$ . Conceptually, any probability distribution on $\mathbb{S}(R')$ is conceivable. In this paper, we follow a simpler approach and select $\pi$ uniformly at random from $\mathbb{S}(R')$ . In this way, we obtain an ordered tuple of elements of $R$ without any repetitions.

The result of these two random processes is a tuple $(\rho _{i_1}, \ldots , \rho _{i_k})$ where $k=|R'|$ and $\rho _{i_j}\in R$ . To simplify notation, we identify $R'$ with this tuple in what follows. Since both random events – dropping and shuffling – are independent, the probability of each such tuple $R'=(\rho _{i_1},\ldots , \rho _{i_k})$ is precisely $\Pr [R'] = \frac {p_d^{n-k}(1-p_d)^k}{k!}$ .

These observations motivate the following Markov chain $\mathcal{M}=(\mathcal{S},(X_n)_{n\in \mathbb{N}_0}, p, \varepsilon )$ . The state set $\mathcal{S}$ now consists of stacks of choice-points – in loosely the same way a Prolog runtime would maintain them. Before formally defining the state set and transition probabilities, we invite the reader to consider a simplified graphical representation of the chain, as given in Figure 2.

Figure 2 gives an overview of the chain. Some details have been omitted or simplified to avoid cluttering the picture. Probabilities are not shown. We have omitted choice points from a higher layer: A state/stack of the form $[H|T][](1,5,2,3)[H|T](1,2,3)$ is thus simply represented as $(1,2,3)$ , omitting the “lower” parts of the stack. Note that these items are implicitly clear from the path to a given node. Moreover, most backtracking arrows have been omitted. Finally, at any given depth, both the node $[H|T][]$ and the node $[H|T]$ each have $\sum _{k=0}^r\binom {r}{k}\cdot k!=\lfloor r!\cdot \mathsf{e} \rfloor$ children.Footnote ² These have also mostly been omitted.

Fig. 2. The Markov chain corresponding to $\mathcal{P}$ in the Drop-and-Shuffle approach. Dashed arrows represent backtracking. Double circled nodes produce an output. Hatched nodes are recursive sub-tree roots, which start an entire infinite sub-tree with the same structure as the whole chain (shown as gray triangles).

We describe the state set via (the language defined by) a regular expression. First we need two auxiliary languages:

\begin{align*} \mathcal{S}_{\mathsf{Sel}} & = \{[], [H|T], [][H|T], [H|T][]\}\\ \mathcal{S}_{\mathsf{Com}} & = \{(y_1,\ldots , y_l)\mid 1\leq l \leq r,\;y_i\in \{1,\ldots , r\}, y_i\neq y_j \text{ for all } i\neq j\} \end{align*}

The set $\mathcal{S}_{\mathsf{Com}}$ corresponds to all ordered subsets of command rules, that is, subsets of $\{1,\ldots , r\}$ . Note that the empty subset is excluded, that is, $()\notin \mathcal{S}_{\mathsf{Com}}$ . The set $\mathcal{S}_{\mathsf{Sel}}$ corresponds to the five probabilistic options the drop-and-shuffle algorithm gives us for resolving goals of the form t(X), including potential choice-points for backtracking. Note that, again, the “empty selection” $()\notin \mathcal{S}_{\mathsf{Sel}}$ is not included.

We can now define the state set $\mathcal{S}=(\mathcal{S}_{\mathsf{Sel}}\times \mathcal{S}_{\mathsf{Com}})^*\cdot (\mathcal{S}_{\mathsf{Sel}}+\varepsilon )$ . The “empty state” $\varepsilon \in \mathcal{S}$ is the initial state of the chain; that is, $\mathsf{Init}=\varepsilon$ . We write the pairs $\langle x,y \rangle \in \mathcal{S}_{\mathsf{Sel}}\times \mathcal{S}_{\mathsf{Com}}$ in angular brackets in order to visually distinguish them and improve readability.

Given a state $\varepsilon \neq s\in \mathcal{S}$ , we may write it as $s=w\cdot x$ or $s=w\cdot \langle x,y \rangle$ with $w\in \mathcal{S}$ , $x\in \mathcal{S}_{\mathsf{Sel}}$ , and $y\in \mathcal{S}_{\mathsf{Com}}$ . Because $\varepsilon \notin \mathcal{S}_{\mathsf{Sel}}$ and also $\varepsilon \notin \mathcal{S}_{\mathsf{Com}}$ , this factorization is unique. We make liberal use of this observation when defining the transition probabilities. We first define transitions that descend further into the tree. These correspond to solid arrows in Figure 2. For any $l\geq 1$ , $w\in \mathcal{S}$ , and $x\in \mathcal{S}_{\mathsf{Sel}}$ :

\begin{align*} p(w,\,w\cdot x) &=\frac {(1-p_d)^2}{2} & x\in \{[H|T][],[][H|T]\}\\[3pt] p(w,\,w\cdot x) &= (1-p_d)p_d & x\in \{[],[H|T]\}\\[3pt] p(w\cdot x,\, w\cdot \langle x, (y_1,\ldots , y_l) \rangle ) &= \frac {p_d^{r-l}(1-p_d)^{l}}{l!} & x \in \{[H|T][], [H|T]\} \end{align*}

Next, we define transitions that correspond to backtracking. These correspond to dashed arrows in Figure 2. To this end we define the operation $\mathsf{Pop}\colon \mathcal{S} \rightarrow \mathcal{S}$ . Intuitively, this operation pops from the stack until we arrive at a previously unpursued choice point. Looking back at Figure 2, it identifies the target of the backtracking arrow. It may be necessary to remove multiple layers of pairs $\langle x,y \rangle$ when backtracking. For example $\mathsf{Pop}(\langle [H|T][],(1,3) \rangle \langle [H|T][],(3) \rangle \langle [H|T],(1) \rangle )=\langle [H|T][],(1,3) \rangle []$ . Formally, we define $\mathsf{Pop}$ recursively with $\mathsf{Pop}(\varepsilon )=\bot$ and:

\begin{align*} &\mathsf{Pop}(w\cdot [][H|T])=w\cdot [H|T] & &\mathsf{Pop}(w\cdot [H|T][])=w\cdot []\\[3pt] &\mathsf{Pop}(w\cdot [])=\mathsf{Pop}(w) & & \mathsf{Pop}(w\cdot [H|T])=\mathsf{Pop}(w)\\[3pt] &\mathsf{Pop}(w\cdot \langle x,(y_1) \rangle )=\mathsf{Pop}(w\cdot x) & & \mathsf{Pop}(w\cdot \langle x,(y_1,y_2,\ldots ,y_l) \rangle )= w\cdot \langle x,(y_2,\ldots ,y_l) \rangle \end{align*}

We can now define all backtracking transitions as follows:

\begin{align*} p(w\cdot x, \mathsf{Pop}(w\cdot x)) &= 1 & x = [] \text{ or } x= [][H|T]\\[3pt] p(w\cdot x, \mathsf{Pop}(w\cdot x)) &= p_d^r & x=[H|T] \text{ or } x=[H|T][]\\[3pt] p(w, \mathsf{Pop}(w)) &= p_d^2 & w=w'\cdot \langle x,y \rangle \text{ or } w=\varepsilon \end{align*}

The last two model the event that all matching rules are dropped when unifying command(X) or t(X), respectively.

Note the recursive structure of $\mathcal{M}$ : Any state of the form $w\cdot \langle x,y \rangle$ ( $x\in \mathcal{S}_{\mathsf{Sel}},\;y\in \mathcal{S}_{\mathsf{Com}}$ ) is the root of an infinite sub-tree that has a structure identical to that of $\mathcal{M}$ . We call such states recursive sub-tree roots or simply sub-tree roots. These states are drawn with hatched background in Figure 2.

To each recursive sub-tree root $s$ corresponds a unique state $\mathsf{Exit}_s$ that is visited when the chain leaves that sub-tree (via backtracking). In other words: All paths that exit the sub-tree below $s$ must traverse $\mathsf{Exit}_s$ . For example, consider the left-most gray sub-tree in Figure 1 (with the large gray triangle in the background) below $s=\langle [H|T][],(1,2,\ldots ,r) \rangle$ . Its exit-state is the next node to the right: $\mathsf{Exit}_s=\langle [H|T][],(2,\ldots ,r) \rangle$ . In general, if $s$ is of the form $w\langle x,y \rangle$ , then $\mathsf{Exit}_s=\mathsf{Pop}(w\cdot \langle x,y \rangle )$ . Note that $\mathsf{Exit}_s$ may be at the same depth as $s$ or at a lower depth than $s$ . It is never at a higher depth.

We can now compute the average number of generated tests as before. It is again the mean hitting time $h^{\bot }_{\varepsilon }$ . By an argument identical to Proposition1, the chain will reach $\bot$ eventually with probability 1, if $p_d \gt 0$ . But that does not imply that the hitting time – an expected value – converges for all such values of $p_d$ . Indeed, the hitting time is finite only for a subset of possible choices for $p_d\gt 0$ , as the following theorem shows:

Theorem 3. Let $p_d\in (1-\frac {1}{\sqrt {r}},1]$ . Then the expected number $h^{\bot }_{\varepsilon }$ of states visited from $\varepsilon$ is finite and given by:

\begin{align*} h^{\bot }_{\varepsilon } = \frac {1 + 2(1-p_d)}{1- r(1-p_d)^2} \end{align*}

Moreover this hitting time is identical to $h^{\mathsf{Exit}_s}_{s}$ for any recursive sub-tree root $s$ and its corresponding exit state $\mathsf{Exit}_s$ . We define $C\stackrel {\text{ def}}{=} h^{\bot }_{\varepsilon }$ and remark that it is a constant property of the chain.

Proof. The fact that $h^{\bot }_{\varepsilon }=h^{\mathsf{Exit}_s}_{s}$ for any recursive sub-tree root $s$ is apparent from the definition of the chain: The transition probabilities are prefix invariant. So $p(x\cdot a, x\cdot b) = p(a,b)$ for all factorizations $s=xa$ with $x\in \mathcal{S}$ . $h^{\bot }_{\varepsilon }$ is the unique minimal positive solution to the equations Eq. (1). Now, letting $C\stackrel {\text{ def}}{=} h^{\bot }_{\varepsilon }$ :

\begin{align*} C &= 1 + p_d(1-p_d)\cdot 1 + p_d^2\cdot 0\\[3pt] &\quad + p_d(1-p_d)\cdot \left (1+ \sum _{k=0}^r \frac {p_d^{r-k}(1-p_d)^k}{k!}\cdot k!\cdot \binom {r}{k}\cdot k\cdot C\right ) \end{align*}

\begin{align*}&\quad + \frac {1}{2}\cdot (1-p_d)^2\cdot \left (2 + \sum _{k=0}^r \frac {p_d^{r-k}(1-p_d)^k}{k!}\cdot k!\cdot \binom {r}{k}\cdot k\cdot C\right )\\[3pt] &\quad + \frac {1}{2}\cdot (1-p_d)^2\cdot \left (1 + \sum _{k=0}^r \frac {p_d^{r-k}(1-p_d)^k}{k!}\cdot k!\cdot \binom {r}{k}\cdot (k\cdot C + 1)\right )\\[-12pt]\nonumber \end{align*}

We recall that $(x+y)^r=\sum _{k=0}^{r}\binom {r}{k}x^k y^{r-k}$ . Differentiation and subsequent multiplication by $x$ gives $rx(x+y)^{r-1} = \sum _{k=0}^rk\binom {r}{k}x^{k}y^{r-k}$ . Moreover, recall that $\sum _{k=0}^{r}\binom {r}{k}p_d^{r-k}(1-p_d)^k=1$ . With this, the above simplifies to

\begin{align*} C = 1 + 2(1-p_d) + Cr(1-p_d)^2 \end{align*}

Now if $p_d\leq 1- \frac {1}{\sqrt {r}}$ , then this implies $C \geq 1 + \frac {2}{\sqrt {r}} + C$ , which is possible only if $C=\infty$ . On the other hand, if $p_d\in (1-\frac {1}{\sqrt {r}},1]$ then solving for $C$ establishes the claim. Note that on this interval, the formula has no singularities and is positive.

3.2.2 Infinite looping and time-to-hit

We again construct a recursive analysis of the mean-hitting-time. Recall that in subsection 3.1.2 we analyzed the hitting time of the unique state corresponding to $\tau =(\tau _1,\ldots ,\tau _l)$ (with $\tau _i\in \{1,\ldots ,r\}$ ) by first considering the hitting time of the unique state corresponding to $\tau ^{(l-1)}=(\tau _1,\ldots , \tau _{l-1})$ , and then constructing the full hitting time from that number “bottom-up”. This recursive argument works less well in the present scenario, where many different states correspond to $\tau ^{(l-1)}$ . Computing the overall hitting time as a sum of those intermediate hitting times is challenging, as we explain below. We therefore develop an approach to compute the hitting time “top-down”.

First, we need to add looping to the Markov chain, as in subsection 3.1.2. Recall that there we merged the two states $\sharp$ and $\bot$ . We do not do that here. Instead, we add an edge from $\bot$ to $\varepsilon$ with probability 1. This is for technical reasons, and we will justify this choice further below.

Fig. 3. Recursive sub-tree roots are connected to all ordered subsets of $\{1,\ldots ,r\}$ (most arrows omitted for readability). Each subset of size $k$ starts chain of $k$ subsets. One element (here $1$ ) is the desired next item of the test sequence, and each chain contains at most one state with the desired item at the top (depicted in green).

When running the program $\mathcal{P}$ in the drop-and-shuffle strategy, first we randomly select and permute a subset of the two rule-heads t([]) and t([H–T]). To proceed, we need to visit $[H|T]$ or $[H|T][]$ at depth 0. Once that happens, there are two options: With probability $p_d$ , the next state does not contain $\tau _{1}$ , and thus we cannot visit $\tau$ in this loop iteration (i.e. before returning to $\varepsilon$ first). We call such a sub-tree root unproductive. Conversely, with probability $(1-p_d)$ , the next state does contain $\tau _{1}$ somewhere within its stack (though not necessarily at the top). Those sub-tree roots and their corresponding sub-trees are called productive (shown in green in Figure 3). If a productive or unproductive state does not have $\tau _{1}$ at its top, there is an infinite tree below it that will we be explored, but cannot yield the desired test case. We also call such a sub-tree unproductive (shown in gray in Figure 3). Note that productive sets of size $k\gt 0$ give rise to precisely $k-1$ unproductive sub-trees, though not all of those are traversed before the sub-tree containing $\tau$ is visited. Let $A_\tau$ denote the set of states that output $\tau$ :

Note that we are only talking about depth 0 for now, so this fact is obvious. At higher depths, we would need to adjust the definition of “productive” to ensure that all prefixes at lower depths have $\tau _1,\tau _2,\ldots$ at the top.

If the chain arrives at a productive state, the situation is more complex: The chain needs to traverse a number of unproductive sub-trees, depending on the position of $\tau _{1}$ in the ordered set. For example, in the second branch shown in Figure 3, there are two unproductive sub-trees to be traversed before reaching productive sub-tree (and several, indicated by the dots, after). Below the productive state, we find a tree of recursive structure: The goal now is to reach a test case of length $l-1$ , or to return to $\varepsilon$ and start from scratch (cf. Fact1).

Starting from a productive state $s$ of size $m\leq r$ with item $\tau _{1}$ at position $k\in \{0,\ldots , m\}$ , the hitting time is thus $kC+h^{A_\tau }_{x}$ , where $C$ is the quantity from Theorem3 and $x\in \mathcal{S}$ is the state arising from $s$ after $k$ items have been popped (e.g. the green state in the lower branch in Figure 3).

We stress that $h^{A_\tau }_{x}$ depends not only on the remaining test-case suffix $(\tau _2,\ldots , \tau _l)$ , but also on the number of (unproductive) backtracking steps – at least $(m-k-1)C$ – before returning to $\varepsilon$ . The chain may arrive at an unproductive step at some point, and by Fact1, it first needs to visit $\varepsilon$ before reaching $A_\tau$ . Note that this may happen at depth $\gt 1$ as well! This prevents us from applying induction in a straightforward way. We thus need the following lemma, which allows us to reduce the hitting time $h^{A_\tau }_{x}$ to three quantities we may compute individually and inductively:

Lemma 2. For any recurrent Markov chain, let $A\subseteq \mathcal{S}$ and $x,y\in \mathcal{S}\setminus A$ with $x\neq y$ . Then:

\begin{align*} h^{A}_{x}=h^{A\cup \{y\}}_{x} + \Pr [H^{y}_{x} \lt H^{A}_{x}]\cdot h^{A}_{y} \end{align*}

We prove the lemma at the end of this section, as it is purely Markov theoretic. In our immediate setting, the consequence is that

(7) \begin{align} h^{A_{\tau }}_{\varepsilon} & =h^{A_{\tau }\cup \{\bot \}}_{\varepsilon } + q\cdot h^{A_{\tau }}_{\bot } =h^{A_{\tau }\cup \{\bot \}}_{\varepsilon } + q(1 + h^{A_{\tau }}_{\varepsilon })\nonumber \\ \iff h^{A_{\tau }}_{\varepsilon } & =\frac {h^{A_{\tau }\cup \{\bot \} }_{\varepsilon }+ q }{1- q} \end{align}

for some probability $q$ that depends on $\tau$ . We shall see below that $q$ in fact only depends on the length $l$ of the test case. Note that by not merging $\bot$ and $\varepsilon$ , we fulfill the “ $x\neq y$ premise” of the lemma. By choosing $p_d\in (1-\frac {1}{\sqrt {r}},1)$ , we obtain a recurrent chain.

It is thus sufficient to compute $q$ and $h^{A_\tau \cup \{\bot \}}_{\varepsilon }$ . We have “widened” the set $A_\tau$ by including $\bot$ , which means we no longer have to worry about looping back to $\varepsilon$ . This effectively allows us to compute the desired hitting-time inductively, applying Lemma2 iteratively. We first turn to computing $q$ as a function of $\tau$ .

Let $s=\langle x_1,y_1 \rangle \cdots \langle x_m,y_m \rangle \in \mathcal{S}$ be any recursive sub-tree root, where $y_i=(y_{i,1},\ldots , y_{i,n_i})\in \mathcal{S}_{\mathsf{Com}}$ for $1\leq i \leq m$ . Then the sequence $y_{1,1}\cdots y_{m,1}$ is the prefix of $s$ . It corresponds to the sequence that would be output if state $s\cdot []$ is reached.

In what follows, we write $\tau ^{(i)} = (\tau _{1},\ldots ,\tau _i)$ . So $\tau =\tau _l$ and $\tau _0=()$ is the empty test sequence. Write $A_i$ for the set of states that output $\tau ^{(i)}$ . For simplicity, we write $h_i=h^{A_i\cup \{\bot \}}_{\varepsilon }$ .

Lemma 3. Let $p_d\in (1-\frac {1}{\sqrt {r}},1)$ and $i\in \{0,\ldots , l\}$ and let $C$ denote the constant from Theorem 3. Then

\begin{align*} h_{i} & = (1-p_d)^{2i}\left(1+\frac {(1-p_d^2)(1+C)}{2} -\frac {C(1-p_d)^2(r-1)}{p_d(2-p_d)}\right.\\ &\quad -\,\left.\frac {(1-p)(1+p) + (1-p)^3}{2p_d(2-p_d)} - i\cdot (1-p_d)^{2}\cdot \frac {1+C(r-1)}{2}\right) \\ &\quad + \left (\frac {C(1-p_d)^2(r-1)}{p_d(2-p_d)} + \frac {(1-p)(1+p) + (1-p)^3}{2p_d(2-p_d)}\right ) \end{align*}

Proof. For $h_0$ we need the hitting time of the set of four states $\bot$ , $[]$ , $[H|T][]$ , and $[][H|T]$ . By Eq. (1) we have

\begin{align*} h_0=1 + (1-p_d)p_d(1+C) + \frac {1}{2}(1-p_d)^2(1+C)= 1+\frac {1}{2}(1-p_d^2)(1+C) \end{align*}

To compute $h_{i+1}$ , we first look at the hitting time starting from the states $[H|T]$ , $[H|T][]$ , and $[][H|T]$ . We select a productive state with probability $(1-p_d)$ and an unproductive one with probability $p_d$ . In either case we traverse some number $0\leq k \leq r-1$ of unproductive sub-trees, each of which adds $C$ steps. In case of a productive state, the number depends on the position of $\tau _1$ in the set of $k+1$ elements:

\begin{align*} h^{A_{i+1^{\cup \{\bot \}}}}_{[H|T]}&=p_d\cdot \sum _{k=0}^{r-1}\binom {r-1}{k}p_d^{r-1-k}(1-p_d)^k \cdot kC & \text{(unproductive)}\\ &\quad+ (1-p_d)\cdot \sum _{k=0}^{r-1}\binom {r-1}{k}\frac {p_d^{r-1-k}(1-p_d)^k}{k+1} & \text{(productive)}\\ &\qquad \cdot \sum _{m=0}^{k}mC + h_{i} + q^{(i)}(m-k)C & \text{(proposition 2 and lemma 2)} \end{align*}

Note the denominator $k+1$ accounts for the probability of placing $\tau _i$ at position $m=0,1,\ldots ,k$ . At position $m$ , there are $mC$ unproductive sub-trees before reaching $\tau _1$ and $(m-k)$ after. In the third line, we split the recursive hitting time using Lemma2. Note that the exit state $\mathsf{Exit}_s$ of each recursive sub-tree root $s$ reached in this way is either $\bot$ (if $k=m$ ) or the adjacent unproductive recursive sub-tree root (cf. Figure 3). By Proposition2 Item 1 we may use recursion.

Computing the inner sum and canceling out the denominator $k+1$ from the second line, we get:

\begin{align*} h^{A_{i+1^{\cup \{\bot \}}}}_{[H|T]}&=p_d\cdot \sum _{k=0}^{r-1}\binom {r-1}{k}p_d^{r-1-k}(1-p_d)^k \cdot kC \end{align*}

\begin{align*}&\qquad + (1-p_d)\cdot \sum _{k=0}^{r-1}\binom {r-1}{k}p_d^{r-1-k}(1-p_d)^k \cdot \left (\frac {kC}{2} + h_i + q^{(i)}\frac {kC}{2}\right ) \\ &\quad= p_d(1-p_d)(r-1)C + (1-p_d)\left (h_i + (1+q^{(i)})\frac {(r-1)(1-p_d)C}{2}\right ) \end{align*}

where the Binomial sums are computed as in the proof of Theorem3. The formulas for the remaining states are completely analogous, but we have additional steps for the intermediate detours over $[]$ . It is thus convenient to express them in terms of $h^{A_{i+1^{\cup \{\bot \}}}}_{[H|T]}$ :

(*) \begin{align} h^{A_{i+1^{\cup \{\bot \}}}}_{[][H|T]}&= 1+ h^{A_{i+1^{\cup \{\bot \}}}}_{[H|T]} \nonumber\\ h^{A_{i+1^{\cup \{\bot \}}}}_{[H|T][]} &=h^{A_{i+1^{\cup \{\bot \}}}}_{[H|T]} + (1-p_d)q^{(i)}\end{align}

For the second identity, note that visits to $[]$ are counted only in case of failure and are thus weighted by $q^{(i)}$ .

We can now turn to $h_{i+1}$ . By Eq. (1):

\begin{align*} h_{i+1}&= p_d^2\cdot 0 + (1-p_d)p_d + (1-p_d)p_d\cdot h^{A_{i+1^{\cup \{\bot \}}}}_{[H|T]} \\ &\quad + \frac {(1-p_d)^2}{2}\cdot \left(h^{A_{i+1^{\cup \{\bot \}}}}_{[][H|T]} + h^{A_{i+1^{\cup \{\bot \}}}}_{[H|T][]}\right) \end{align*}

which, after substitution of ( $\ast$ ) and straightforward simplification becomes

\begin{align*} h_{i+1}= \frac {1}{2} \left (1-p_d^2 + (1-p_d)^3q^{(i)}\right ) + (1-p_d)h^{A_{i+1^{\cup \{\bot \}}}}_{[H|T]} \end{align*}

We substitute $h^{A_{i+1}}_{[H|T]}$ and Proposition2 Item 2, then simplify, isolating the terms that are dependent on $i$ :

\begin{align*} h_{i+1} & = (1-p_d)^2h_i -\frac {1+C(r-1)}{2}\cdot (1-p_d)^{2i+4} \\ &\quad + C(1-p_d)^2(r-1) + \frac {(1-p)(1+p) + (1-p) ^3}{2} \end{align*}

With $\alpha =(1-p_d)^2$ , $\beta = -\frac {1+C(r-1)}{2}\cdot (1-p_d)^4$ , and $\gamma =C(1-p_d)^2(r-1) + \frac {(1-p)(1+p) + (1-p)^3}{2}$ we get the following formula (e.g. by iterative substitution):

(8) \begin{equation} h_{i+1} = \alpha ^{i+1}h_0 + (i+1) \beta \alpha ^i + \gamma \,\frac {1-\alpha ^{i+1}}{1-\alpha } \end{equation}

Its correctness is easily shown by induction on $i\geq 0$ .

Substituting $h_0$ , $\alpha$ , $\beta$ , and $\gamma$ into Eq. (8) gives:

\begin{align*} h_{i} & = (1-p_d)^{2i}h_0 - i\cdot (1-p_d)^{2(i-1)+4}\cdot \frac {1+C(r-1)}{2} \\&\quad + \left (C(1-p_d)^2(r-1) + \frac {(1-p)(1+p) + (1-p)^3}{2}\right )\frac {1-(1-p_d)^{2i}}{1-(1-p_d)^2} \end{align*}

which simplifies to the desired formula.

Substituting into Eq. (7) gives the following theorem:

Theorem 4. Let $p_d\in (1-\frac {1}{\sqrt {r}},1)$ , $r\in \mathbb{N}$ . Denote by $C$ the constant from Theorem 3. Let $\tau$ be a test case of length $l$ and $A_\tau \subseteq \mathcal{S}$ the set of states that output $\tau$ . Then:

\begin{align*} h^{A_{\tau }}_{\varepsilon} & =\frac {1+ \left (\frac {C(1-p_d)^2(r-1)}{p_d(2-p_d)} + \frac {(1-p)(1+p) + (1-p)^3}{2p_d(2-p_d)}\right ) }{(1-p_d)^{2l+1}} \\ &\quad + \frac {(1+p_d)(1+C)}{2} -\frac {C(1-p_d)(r-1)}{p_d(2-p_d)} - \frac {(1+p_d) + (1-p_d)^2}{2p_d(2-p_d)} \\ &\quad - l\cdot (1-p_d)\cdot \frac {1+C(r-1)}{2} - (1-p_d) +\frac {1}{(1-p_d)} \end{align*}

All that remains is to prove Lemma2:

Proof of Lemma 2.

\begin{align*} \nonumber\\[-30pt]h^{A}_{x} &= \sum _{n=0}^{\infty }n\cdot \Pr [H^{A}_{x}=n,H^{A}_{x} \lt H^{y}_{x}] + \sum _{n=0}^{\infty }\sum _{k=0}^{n-1}n\cdot \Pr [H^{A}_{x}=n,H^{A}_{x}\gt H^{y}_{x},H^{y}_{x}=k] \end{align*}

Using Fubini’s theorem, the second sum becomes

\begin{align*} &\sum _{k=0}^{\infty }\Pr [H^{y}_{x}=k,H^{A}_{x}\gt H^{y}_{x}]\sum _{n=k+1}^{\infty }n\cdot \Pr [H^{A}_{x}=n|H^{y}_{x}=k,H^{A}_{x}\gt H^{y}_{x}]\\ &\quad = \sum _{k=0}^{\infty }\Pr [H^{y}_{x}=k,H^{A}_{x}\gt H^{y}_{x}]\sum _{m=1}^{\infty }(m+k)\Pr [H^{A}_{y}=m] \end{align*}

and since $\Pr [H^{A}_{y}=0]$ , because $y\notin A$ , and moreover $\mathcal{M}$ is recurrent (so $\Pr [H^{A}_{y}\lt \infty ]=1$ ) this is equal to

\begin{align*} & \sum _{k=0}^{\infty }\Pr [H^{y}_{x}=k,H^{A}_{x}\gt H^{y}_{x}]\left (k + h^{A}_{y}\right ) \\&\quad = \left (\sum _{k=0}^{\infty }\Pr [H^{y}_{x}=k,H^{A}_{x}\gt H^{y}_{x}]k \right )+ h^{A}_{y} \cdot \Pr [H^{A}_{x}\gt H^{y}_{x}]. \end{align*}

Because $y\notin A$ , we have $H^{A}_{x}\neq H^{y}_{x}$ and so $\Pr [H^{y}_{x}=k,H^{A}_{x}\gt H^{y}_{x}] + \Pr [H^{A}_{x}=k,H^{A}_{x}\lt H^{y}_{x}]=\Pr [H^{A\cup \{y\}}_{x}=k]$ . Hence the remaining two infinite sums add up to $h^{A\cup \{y\}}_{x}$ .