2. Preliminaries

In this section, some basic knowledge on the theory of aggregated Markov process are presented, which are developed in pathbreaking papers such as Colquhoun and Hawkes [Reference Colquhoun and Hawkes11] in the ion channel literature and other papers like Ball and Sansom [Reference Ball and Sansom5] in the probability literature and Rubino and Sericola [Reference Rubino and Sericola24] and Zheng et al. [Reference Zheng, Cui and Hawkes31] in the reliability literature. These knowledge form our basic concepts and notations in this paper. The basic assumptions to be used are also discussed, and the proofs of some of them are given in this section.

Consider a finite-state homogenous continuous-time Markov process $\{X(t),t \ge 0\}$ with transition rate matrix ${\bf{Q}} = {\left( {{q_{ij}}} \right)_{n \times n}}$, state space ${\bf{S}} = \{1,2, \ldots ,n\} $, and initial probability vector ${{\bf{\alpha }}_0} = ({\alpha _1},{\alpha _2}, \ldots ,{\alpha _n})$. The stochastic process $\{X(t),t \ge 0\}$ can result from many areas such as ion channel, quality and reliability, operational management, and so on. With many possible reasons, the state space can be assumed to be aggregated by partitioning into classes, so that it is possible to observe only which class the stochastic process is in at any given time instead the specific state it is in. This fact forms a new stochastic process $\{\tilde X(t),t \ge 0\} $ with state space ${\tilde{\bf{S}}}$ such that $|{\tilde{\bf{S}}}| \lt |{\bf{S}}|$, that is, each class in S is a state in ${\tilde{\bf{S}}}$. The stochastic process $\{\tilde X(t),t \ge 0\} $ is called an aggregated Markov process, and the stochastic process $\{X(t),t \ge 0\} $ is called the underlying one. In reliability field, in general, the state space S can be divided into two classes: the working (up) class, denoted by W, and the failure (down) class, denoted by F, that is, ${\bf{S}} = {\bf{W}} \cup {\bf{F}}$. Without loss of generality, it is assumed that ${\bf{W}} = \{1,2, \ldots ,{n_o}\}$ and ${\bf{F}} = \{{n_o} + 1,{n_o} + 2, \ldots ,n\}$. The corresponding transition rate matrix Q can also be written as

(2.1)\begin{equation} {\bf{Q}} = \left( { \begin{matrix} {{{\bf{Q}}_{{\bf{WW}}}}} & {{{\bf{Q}}_{{\bf{WF}}}}} \\ {{{\bf{Q}}_{{\bf{FW}}}}} & {{{\bf{Q}}_{{\bf{FF}}}}} \end{matrix} } \right). \end{equation}

To study the aggregated Markov process, the following concepts and notation are given:

(2.2)\begin{align} {}^{\bf{W}}{p_{ij}}(t): = P\{& X(t){\rm{~remains~within~}}{\bf{W}}{\rm{~throughout~time~0~to~time~}}t, \nonumber\\ & {\rm{and~is~in~state~}}j{\rm{~at~time~}}t|X(0) = i\}, \quad{\rm}i,j \in {\bf{W}}. \end{align}

After simple manipulations, we get, in matrix form, ${}^{\bf{W}}{p_{ij}}(t)$ are the elements of the ${n_o} \times {n_o}$ matrix

(2.3)\begin{equation} {{\bf{P}}_{{\bf{WW}}}}(t) = {\left( {{}^{\bf{W}}{p_{ij}}(t)} \right)_{{n_o} \times {n_o}}} = \exp ({{\bf{Q}}_{{\bf{WW}}}}t). \end{equation}

Another important quantity is defined as

(2.4)\begin{align} {g_{ij}}(t): = \mathop {\lim }\limits_{\Delta t \to 0} [P\{& X(t){\rm{~stays~in~}}{\bf{W}}{\rm{~from~time~0~to~time~}}t,{\rm{~and~leaves~}}{\bf{W}}{\rm{~for~state~}}j \nonumber\\ & {\rm{~between~}}t{\rm{~and~}}t + \Delta t|X(0) = i\} /\Delta t], \quad{\rm}i \in {\bf{W}},j \in {\bf{F}}. \end{align}

Similarly, we have, in matrix form, ${g_{ij}}(t)$ are the elements of the ${n_o} \times (n - {n_o})$ matrix

(2.5)\begin{equation} {{\bf{G}}_{{\bf{WF}}}}(t) = {{\bf{P}}_{{\bf{WW}}}}(t){{\bf{Q}}_{{\bf{WF}}}}. \end{equation}

The Laplace transform will be used throughout the paper, which is defined for function f(t) as ${f^*}(s) = \int_0^\infty {{{\rm e}^{- st}}f(t)\,{\rm{d}}t}$. The Laplace transform for a functional matrix is defined elementwise. The Laplace transforms of Eqs. (2.3) and (2.5), respectively, are

(2.6)\begin{align} {\bf{P}}_{{\bf{WW}}}^*(s) = {(s{\bf{I}} - {{\bf{Q}}_{{\bf{WW}}}})^{- 1}}, \end{align}

(2.7)\begin{align} {\bf{G}}_{{\bf{WF}}}^*(s) = {\bf{P}}_{{\bf{WW}}}^*(s){{\bf{Q}}_{{\bf{WF}}}} = {(s{\bf{I}} - {{\bf{Q}}_{{\bf{WW}}}})^{- 1}}{{\bf{Q}}_{{\bf{WF}}}}, \end{align}

${\bf{G}}_{{\bf{WF}}}^*(0)$

${{\bf{G}}_{{\bf{WF}}}}$

(2.8)\begin{equation} {{\bf{G}}_{{\bf{WF}}}} = {\bf{G}}_{{\bf{WF}}}^*(0) = - {\bf{Q}}_{{\bf{WW}}}^{- 1}{{\bf{Q}}_{{\bf{WF}}}}. \end{equation}

The inversion of a partition matrix will be used in this paper; the related results are presented as follows.

Given a four-block partitioned matrix

\begin{align*} {\bf{M}} = \left( \begin{matrix} {{{\bf{M}}_{{\bf{WW}}}}} & {{{\bf{M}}_{{\bf{WF}}}}} \\ {{{\bf{M}}_{{\bf{FW}}}}} & {{{\bf{M}}_{{\bf{FF}}}}} \end{matrix} \right), \end{align*}

if ${{\bf{M}}_{{\bf{WW}}}}$ and ${{\bf{M}}_{{\bf{FF}}}}$ are not singular, as discussed in Colquhoun and Hawkes [Reference Colquhoun and Hawkes11], its inversed matrix is

\begin{equation*} {{\bf{M}}^{- 1}} = \left( \begin{matrix} {{{\bf{X}}_{\bf{W}}}} & {- {\bf{M}}_{{\bf{WW}}}^{- 1}{{\bf{M}}_{{\bf{WF}}}}{{\bf{X}}_{\bf{F}}}} \\ {- {\bf{M}}_{{\bf{FF}}}^{- 1}{{\bf{M}}_{{\bf{FW}}}}{{\bf{X}}_{\bf{W}}}} & {{{\bf{X}}_{\bf{F}}}} \end{matrix} \right), \end{equation*}

where

\begin{align*} &{{\bf{X}}_{\bf{W}}} = {({\bf{I}} - {\bf{M}}_{{\bf{WW}}}^{- 1}{{\bf{M}}_{{\bf{WF}}}}{\bf{M}}_{{\bf{FF}}}^{- 1}{{\bf{M}}_{{\bf{FW}}}})^{- 1}}{\bf{M}}_{{\bf{WW}}}^{- 1},\\ &{{\bf{X}}_{\bf{F}}} = {({\bf{I}} - {\bf{M}}_{{\bf{FF}}}^{- 1}{{\bf{M}}_{{\bf{FW}}}}{\bf{M}}_{{\bf{WW}}}^{- 1}{{\bf{M}}_{{\bf{WF}}}})^{- 1}}{\bf{M}}_{{\bf{FF}}}^{- 1}. \end{align*}

Using the results presented above, for

\begin{align*} (s{\bf{I}} - {\bf{Q}}) = \left( \begin{matrix} {s{\bf{I}} - {{\bf{Q}}_{{\bf{WW}}}}} & {- {{\bf{Q}}_{{\bf{WF}}}}} \\ {- {{\bf{Q}}_{{\bf{FW}}}}} & {s{\bf{I}} - {{\bf{Q}}_{{\bf{FF}}}}} \end{matrix} \right), \end{align*}

we have

(2.9)\begin{align} {(s{\bf{I}} - {\bf{Q}})^{- 1}} = \left( \begin{matrix} {{{[{\bf{P}}(s)]}_{{\bf{WW}}}}} & {{{[{\bf{P}}(s)]}_{{\bf{WF}}}}} \\ {{{[{\bf{P}}(s)]}_{{\bf{FW}}}}} & {{{[{\bf{P}}(s)]}_{{\bf{FF}}}}} \end{matrix} \right), \end{align}

where

\begin{align*} {[{\bf{P}}(s)]_{{\bf{WW}}}} &= {[{\bf{I}} - {\bf{G}}_{{\bf{WF}}}^*(s){\bf{G}}_{{\bf{FW}}}^*(s)]^{- 1}}{\bf{P}}_{{\bf{WW}}}^*(s),\\ {[{\bf{P}}(s)]_{{\bf{FW}}}} & = {\bf{G}}_{{\bf{FW}}}^*(s){[{\bf{I}} - {\bf{G}}_{{\bf{WF}}}^*(s){\bf{G}}_{{\bf{FW}}}^*(s)]^{- 1}}{\bf{P}}_{{\bf{WW}}}^*(s). \end{align*}

For the results on the inversion of a partitioned matrix, there is a requirement on the existence of both matrix ${(s{\bf{I}} - {{\bf{Q}}_{{\bf{WW}}}})^{- 1}}$ and matrix ${(s{\bf{I}} - {{\bf{Q}}_{{\bf{FF}}}})^{- 1}}$ for any variable s > 0. The proof for these can be found in several literature, for example, see Yin and Cui [Reference Yin and Cui30].

On the other hand, the existence of ${\bf{Q}}_{{\bf{WW}}}^{- 1}$ and ${\bf{Q}}_{{\bf{FF}}}^{- 1}$ is also needed for the underlying stochastic Markov process $\{X(t),t \ge 0\} $. In the ion channel modeling literature, the time reversibility and communication for all states on the underlying process are assumed, but in our paper, it is extended into the situation of ${\rm{Rank}}({\bf{Q}}) = n - 1$, that is, the rank of transition rate matrix Q is equal to n − 1, which can not only cover the case in ion channel modeling but also include many other cases, for example, see some numerical examples presented in Section 5 of this paper. The proof of the existence of ${\bf{Q}}_{{\bf{WW}}}^{- 1}$ and ${\bf{Q}}_{{\bf{FF}}}^{- 1}$ is given in the following lemma.

Proof. Let the transition rate matrix be

\begin{equation*}\bf Q={\left(\begin{array}{@{\kern-0.1pt}cccc@{\kern-0.1pt}}-q_{11}&q_{12}&\cdots&q_{1n}\\q_{21}&-q_{22}&\ddots&\vdots\\\vdots&\ddots&\ddots&q_{(n-1)n}\\q_{n1}&\cdots&q_{n(n-1)}&-q_{nn}\end{array}\right)}\end{equation*}

and the steady-state probability vector be ${\bf{\pi }} = {({\pi _1},{\pi _2}, \ldots ,{\pi _n})^{\rm T}}$. Since the vector π satisfies the following set of equations

\begin{equation*}\left \{\begin{array}{@{\kern-0.1pt}c}\bf Q^{\mathrm T}\bf\pi=\mathbf0,\\\sum_{i=1}^n\pi_i=1,\end{array}\right.\end{equation*}

which is equivalent to the following linear set of equations

(2.10)\begin{equation} {{\tilde{\bf{Q}}}_i}{\bf{\pi }} = {{\bf{b}}_i}, \end{equation}

where ${{\bf{b}}_i} = {(\underbrace {0, \ldots ,0,1,}_i0, \ldots ,0)^{\rm T}}$ and

\begin{equation*}\begin{array}{rlrlrlrlrlrl}{\widetilde{\bf Q}}_i={\left(\begin{array}{@{\kern-0.1pt}cccc@{\kern-0.1pt}} -q_{11}& q_{21} & \cdots & q_{n1}\\ \vdots & \vdots & \ddots & \vdots\\ q_{1(i-1)} & q_{2(i-1)} & \cdots & q_{n(i-1)}\\ 1 & 1 & \cdots & 1\\ q_{1(i+1)} & q_{2(i+1)} & \cdots & q_{n(i+1)}\\ \vdots & \vdots & \ddots & \vdots\\ q_{1n} & q_{2n} & \cdots & -q_{nn}\end{array}\right)}.\end{array}\end{equation*}

Because we know that there exists a unique solution π to Eq. (2.10) and all π_i are greater than zero in terms of all states being communicated to each other, then based on Cramer rule, we have

\begin{align*} &{\rm{Rank}}({{\tilde{\bf{Q}}}_i}) = {\rm{Rank}}({{\tilde{\bf{Q}}}_i};{{\bf{b}}_i}) = n, \\ &{\pi _j} = {{\det [{{{\tilde{\bf{Q}}}}_{i,j}}]} \over {\det [{{{\tilde{\bf{Q}}}}_i}]}},\quad j = 1,2, \ldots ,n, \end{align*}

where ${{\tilde{\bf{Q}}}_{i,j}}$ is a matrix resulting from replacing the jth column of ${{\tilde{\bf{Q}}}_i}$ by vector ${{\bf{b}}_i}$. Furthermore, we have $\det [{{\tilde{\bf{Q}}}_{i,j}}] \ne 0$ because ${\pi _j} \gt 0$, that is, ${\rm{Rank}}({{\tilde{\bf{Q}}}_{i,j}}) = n$ for any $i,j \in {\bf{S}}$. When taking $i = j = n$ as a special case, we have

\begin{equation*}{\widetilde{\bf Q}}_{n-1,n-1}={\left(\begin{array}{@{\kern-0.1pt}cccc@{\kern-0.1pt}}-q_{11}&\cdots&q_{(n-1)1}&0\\\vdots&\ddots&\vdots&\vdots\\q_{1(n-1)}&\cdots&-q_{(n-1)(n-1)}&0\\1&\cdots&1&1\end{array}\right)}\end{equation*}

and

\begin{equation*}\begin{array}{rlrlrlrlrlrl}\operatorname{det}{\left({\widetilde{\bf Q}}_{(n-1),(n-1)}\right)}& =\operatorname{det}{\left(\begin{array}{@{\kern-0.1pt}ccc@{\kern-0.1pt}} - q_{11} & \cdots& q_{(n-1)1}\\\vdots & \ddots & \vdots\\q_{1(n-1)} & \cdots & -q_{(n-1)(n-1)}\end{array}\right)}\\[18pt] & =\operatorname{det}{\left(\begin{array}{@{\kern-0.1pt}ccc@{\kern-0.1pt}} -q_{11} & \cdots & q_{1(n-1)}\\ \vdots& \ddots & \vdots\\ q_{(n-1)1} & \cdots & -q_{(n-1)(n-1)}\end{array}\right)}\neq 0,\end{array}\end{equation*}

that is, ${\rm{Rank}}({\bf{Q}}) = n - 1$. Furthermore, without loss of generality, it is assumed that the state space S can be partitioned into two disjoint parts, that is, ${\bf{S}} = {\bf{W}} + {\bf{F}} = \{1,2, \ldots ,{n_o}\} + \{{n_o} + 1,{n_o} + 2, \ldots ,n\} .$ We consider the special case of $i = j = {n_o} + 1$, that is,

\begin{equation*}{\widetilde{\bf Q}}_{n_o+1,n_o+1}=\begin{pmatrix}-q_{11}&\cdots&q_{n_o1}&0&\cdots\\\vdots&\ddots&\vdots&\vdots&\ddots\\q_{1n_o}&\cdots&-q_{n_on_o}&0&\cdots\\1&\cdots&1&1&\cdots\\q_{1(n_o+2)}&\cdots&q_{1(n_o+2)}&0&\cdots\\\vdots&\ddots&\vdots&\vdots&\ddots\\q_{1n}&\cdots&q_{1n}&0&\cdots\end{pmatrix}=\begin{pmatrix}&&&0&\cdots\\&\bf Q_{\bf W\bf W}^{\mathrm T}&&\vdots&\ddots\\&&&0&\cdots\\1&\cdots&1&1&\cdots\\q_{1(n_o+2)}&\cdots&q_{1(n_o+2)}&0&\cdots\\\vdots&\ddots&\vdots&\vdots&\ddots\\q_{1n}&\cdots&q_{1n}&0&\cdots\end{pmatrix}.\end{equation*}

Thus, we have $\det [{\bf{Q}}_{{\bf{WW}}}^{\rm T}] = \det [{{\bf{Q}}_{{\bf{WW}}}}] \ne 0$, that is to say, ${\rm{Rank}}({{\bf{Q}}_{{\bf{WW}}}}) = {n_o} = |{\bf{W}}|$. Similarly, we can prove that ${\rm{Rank}}({{\bf{Q}}_{{\bf{FF}}}}) = n - {n_o} = |{\bf{F}}|$. It completes the proof.

As mentioned above, if ${\rm{Rank}}({{\bf{Q}}_{n \times n}}) = n - 1$ for a finite-state time-homogenous Markov process, all states may communicate or not, for example, see examples presented in Section 5. On the other hand, Lemma 2.1 told us that if all states communicate with each other in a finite state Markov process, then the rank of its transition rate matrix is n − 1.

Unlike in the ion channel modeling, in this paper, we focus on the limiting behaviors of the aggregated Markov processes, which are used to describe the maintenance processes in the reliability field. The four steady-state reliability indexes are studied, which include the steady-state availability, the steady-state interval availability, the steady-state transition probability between two subsets, and the steady-state probability staying at a subset. All these four indexes can be expressed through using the aggregated Markov process and the initial conditions of the underlying process being at the initial time t = 0, which all are the products of the initial probability vectors and matrices. It is not easy to know directly in terms of these formulas that these expressions do not depend on the initial probability vectors. However, the common knowledge in Markov processes tells us that they hold true. The proofs on that will be presented in terms of these products when time approaches to infinity in next section.

3. Limiting results derived by using aggregated Markov processes

The limiting results in aggregated stochastic processes are considered in many theoretical and practical situations. In reliability field, especially for repairable systems, some limiting behaviors need to be considered. The following four reliability-related measures, in general, are paid attention to:

1. (1) the steady-state availability, denoted as $\mathop {\lim }\limits_{t \to \infty } A(t)$;

2. (2) the steady-state interval availability, denoted as $\mathop {\lim }\limits_{t \to \infty } A([t,t + a])$, where $a \ge 0$;

3. (3) the steady-state transition probability between two subsets, denoted as $\mathop {\lim }\limits_{t \to \infty } {p_{{{\bf{S}}_1}{{\bf{S}}_2}}}(t)$, where ${{\bf{S}}_i} \subsetneq {\bf{S}}$, $(i = 1,2)$ and ${{\bf{S}}_1} \cap {{\bf{S}}_2} = \phi $, ϕ is an empty set; and

4. (4) the steady-state probability staying in a given subset $\mathop {\lim }\limits_{t \to \infty } {p_{{{\bf{S}}_0}}}(t)$, where ${{\bf{S}}_0} \subsetneq {\bf{S}}$, that is, ${{\bf{S}}_0}$ is a proper subset of state space S.

In the sequeal, the computation formulas for the four measures are derived in terms of the theory of aggregated Markov processes.

(1) The steady-state availability

The definition of steady-state availability is given by

(3.1)\begin{equation} \mathop {\lim }\limits_{t \to \infty } A(t) = \mathop {\lim }\limits_{t \to \infty } P\{X(t) \in {\bf{W}}\} . \end{equation}

Let ${\bf{A}}(t) = ({A_1}(t), \ldots ,{A_{|{\bf{W}}|}}(t))$, where ${A_i}(t) = P\{X(t) = i \in {\bf{W}}\} $, then we have

(3.2)\begin{equation} {\bf{A}}(t) = {{\bf{\alpha }}_0}\int\limits_0^t {{\bf{f}}(u){{\bf{P}}_{{\bf{WW}}}}(t - u)\,{\rm{d}}u}, \end{equation}

where ${{\bf{P}}_{{\bf{WW}}}}(t) = \exp ({{\bf{Q}}_{{\bf{WW}}}}t)$ and ${\bf{f}}(t) = {\left( {{f_{ij}}(t)} \right)_{|{\bf{S}}| \times |{\bf{W}}|}}$, its elements ${f_{ij}}(t)$ are the probability density functions of the durations starting from state i at time 0 and ending at time t by entering state j, with initial probability vector ${{\bf{\alpha }}_0} = ({{\bf{\alpha }}_{\bf{W}}},{{\bf{\alpha }}_{\bf{F}}})$. Thus, the Laplace transform of ${\bf{A}}(t)$ is

(3.3)\begin{equation} {{\bf{A}}^*}(s) = {\bf{A}}_{\bf{W}}^*(s) + {\bf{A}}_{\bf{F}}^*(s), \end{equation}

where ${A_{\bf{W}}}(t): = P\{X(t) \in {\bf{W}}\} $ and ${A_{\bf{F}}}(t): = P\{X(t) \in {\bf{F}}\} $,

(3.4)\begin{align} {\bf{A}}_{\bf{W}}^*(s)& = {{\bf{\alpha }}_{\bf{W}}}\sum\limits_{r = 0}^\infty {{{\left[{\bf{G}}_{{\bf{WF}}}^*(s){\bf{G}}_{{\bf{FW}}}^*(s)\right]}^r}} {(s{\bf{I}} - {{\bf{Q}}_{{\bf{WW}}}})^{- 1}}\notag \\ & = {{\bf{\alpha }}_{\bf{W}}}{\left[{\bf{I}} - {\bf{G}}_{{\bf{WF}}}^*(s){\bf{G}}_{{\bf{FW}}}^*(s)\right]^{- 1}}{(s{\bf{I}} - {{\bf{Q}}_{{\bf{WW}}}})^{- 1}} , \end{align}

this is because the system is in W at time t after it spends a duration of either 0, or 1, or 2, $\ldots$ transitions from W to F and back, and the convolution forms a product by taking the Laplace transform. Similarly, we have

(3.5)\begin{align} {\bf{A}}_{\bf{F}}^*(s)& = {{\bf{\alpha }}_{\bf{F}}}{\bf{G}}_{{\bf{FW}}}^*(s)\sum\limits_{r = 0}^\infty {{{\left[{\bf{G}}_{{\bf{WF}}}^*(s){\bf{G}}_{{\bf{FW}}}^*(s)\right]}^r}} {(s{\bf{I}} - {{\bf{Q}}_{{\bf{WW}}}})^{- 1}}\notag \\ & = {{\bf{\alpha }}_F} {\bf{G}}_{{\bf{FW}}}^*(s){\left[{\bf{I}} - {\bf{G}}_{{\bf{WF}}}^*(s){\bf{G}}_{{\bf{FW}}}^*(s)\right]^{- 1}}{(s{\bf{I}} - {{\bf{Q}}_{{\bf{WW}}}})^{- 1}}. \end{align}

Based on Tauber’s Theorem ([Reference Korevaar22], Final Value Theorem [Reference Cohen9]), Theorem 14.1 in reference [Reference Korevaar22], or Theorem 2.6 in reference [Reference Cohen9] tells us that

\begin{equation*}\mathop {\lim }\limits_{t \to 0} t\int\limits_0^\infty {{{\rm e}^{- tv}}} s(v)\,{\rm{d}}v = \mathop {\lim }\limits_{v \to \infty } s(v).\end{equation*}

Then, we have

(3.6)\begin{equation} \mathop {\lim }\limits_{t \to \infty } A(t) = \mathop {\lim }\limits_{t \to \infty } [({{\bf{A}}_W}(t) + {{\bf{A}}_F}(t)){{\bf{u}}_{\bf{W}}}] = \mathop {\lim }\limits_{s \downarrow 0} [s({\bf{A}}_W^*(s) + {\bf{A}}_F^*(s)){{\bf{u}}_{\bf{W}}}], \end{equation}

where ${{\bf{u}}_{\bf{W}}} = (\underbrace {1, \ldots ,1}_{\left| {\bf{W}} \right|})^{\rm T}$.

(2) The steady-state interval availability $\mathop {\lim }\limits_{t \to \infty } A([t,t + a])$

The definition of steady-state interval availability is given by

(3.7)\begin{equation} \mathop {\lim }\limits_{t \to \infty } A([t,t + a]) = \mathop {\lim }\limits_{t \to \infty } P\{X(u) \in {\bf{W}}, {\rm{~for~all~}}u \in [t,t + a]\}. \end{equation}

Since $A([t,t + a]) = {{\bf{A}}_{\bf{W}}}(t)\,\exp (a{{\bf{Q}}_{{\bf{WW}}}}){{\bf{u}}_{\bf{W}}}$, thus we have

(3.8)\begin{align} \mathop {\lim }\limits_{t \to \infty } A([t,t + a]) &= \mathop {\lim }\limits_{t \to \infty } {{\bf{A}}_{\bf{W}}}(t)\,\exp (a{{\bf{Q}}_{{\bf{WW}}}}){{\bf{u}}_{\bf{W}}}\notag \\ &= \mathop {\lim }\limits_{s \to {0^ + }} [s{\bf{A}}_{\bf{W}}^*(s)]\,\exp (a{{\bf{Q}}_{{\bf{WW}}}}){{\bf{u}}_{\bf{W}}}. \end{align}

(3) The steady-state transition probability between two subsets $\mathop {\lim }\limits_{t \to \infty } {p_{{{\bf{S}}_1}{{\bf{S}}_2}}}(t)$

The definition of steady-state transition probability between two subsets is given by

(3.9)\begin{equation} \mathop {\lim }\limits_{t \to \infty } {p_{{{\bf{S}}_1}{{\bf{S}}_2}}}(t) = \mathop {\lim }\limits_{t \to \infty } P\{X(t) \in {{\bf{S}}_2}|X(0) \in {{\bf{S}}_1}\}. \end{equation}

There are two cases to be considered in the following:

Case 1: ${{\bf{S}}_1} \cup {{\bf{S}}_2} = {\bf{S}}$

Similar to Eq. (3.5), we have

(3.10)\begin{align} p_{{{\bf{S}}_1}{{\bf{S}}_2}}^*(s) &= {{\bf{\beta }}_1}{\bf{G}}_{{{\bf{S}}_1}{{\bf{S}}_2}}^*(s)\sum\limits_{r = 0}^\infty {{{[{\bf{G}}_{{{\bf{S}}_2}{{\bf{S}}_1}}^*(s){\bf{G}}_{{{\bf{S}}_1}{{\bf{S}}_2}}^*(s)]}^r}} {\bf{P}}_{{{\bf{S}}_2}{{\bf{S}}_2}}^*(s){{\bf{u}}_{{{\bf{S}}_2}}} \notag \\ & = {{\bf{\beta }}_1}{\bf{G}}_{{{\bf{S}}_1}{{\bf{S}}_2}}^*(s){[{\bf{I}} - {\bf{G}}_{{{\bf{S}}_2}{{\bf{S}}_1}}^*(s){\bf{G}}_{{{\bf{S}}_1}{{\bf{S}}_2}}^*(s)]^{- 1}}{(s{\bf{I}} - {{\bf{Q}}_{{{\bf{S}}_2}{{\bf{S}}_2}}})^{- 1}}{{\bf{u}}_{{{\bf{S}}_2}}}, \end{align}

where the initial probability vector ${{\bf{\beta }}_1} = \left({{{\alpha _1}} \over {\sum\limits_{i \in {{\bf{S}}_1}} {{\alpha _i}} }}, \ldots ,{{{\alpha _{|{{\bf{S}}_1}|}}} \over {\sum\limits_{i \in {{\bf{S}}_1}} {{\alpha _i}} }}\right): = ({\beta _1}, \ldots ,{\beta _{|{{\bf{S}}_1}|}})$ and ${{\bf{u}}_{{{\bf{S}}_2}}} = (\underbrace {1, \ldots ,1}_{\left| {{{\bf{S}}_2}} \right|})$.

Case 2: ${{\bf{S}}_1} + {{\bf{S}}_2} \subsetneq {\bf{S}}$

Let ${{\bar{\bf{S}}}_1} = {\bf{S}}/{{\bf{S}}_1}$, then we first consider the $p_{{{\bf{S}}_1}{{{\bf{\bar S}}}_1}}^*(s)$. From Case 1, we have known that

\begin{equation*} p_{{{\bf{S}}_1}{{{\bar{\bf{S}}}}_1}}^*(s) = {{\bf{\beta }}_1}{\bf{G}}_{{{\bf{S}}_1}{{{\bf{\bar S}}}_1}}^*(s){[{\bf{I}} - {\bf{G}}_{{{{\bf{\bar S}}}_1}{{\bf{S}}_1}}^*(s){\bf{G}}_{{{\bf{S}}_1}{{{\bf{\bar S}}}_1}}^*(s)]^{- 1}}{(s{\bf{I}} - {{\bf{Q}}_{{{{\bf{\bar S}}}_1}{{{\bf{\bar S}}}_1}}})^{- 1}}{{\bf{u}}_{{{{\bf{\bar S}}}_1}}}, \end{equation*}

where ${{\bf{u}}_{{{{\bf{\bar S}}}_1}}}=(\underbrace {1, \ldots ,1}_{\left| {{{{\bar{\bf{S}}}}_1}} \right|})$. On the other hand, we have

(3.11)\begin{equation} p_{{{\bf{S}}_1}{{\bf{S}}_2}}^*(s) = {{{\bf{\beta }}_1}{\bf{G}}_{{{\bf{S}}_1}{{{\bf{\bar S}}}_1}}^*(s){[{\bf{I}} - {\bf{G}}_{{{{\bf{\bar S}}}_1}{{\bf{S}}_1}}^*(s){\bf{G}}_{{{\bf{S}}_1}{{{\bar{\bf{S}}}}_1}}^*(s)]^{- 1}}{(s{\bf{I}} - {{\bf{Q}}_{{{{\bar{\bf{S}}}}_1}{{{\bar{\bf{S}}}}_1}}})^{- 1}}{{\bf{v}}_1}}, \end{equation}

where ${{\bf{v}}_1} = {(\underbrace {1, \ldots ,1}_{\left| {{S_2}} \right|},\underbrace {0, \ldots ,0}_{\left| {{{\bar S}_1}} \right| - \left| {{S_2}} \right|})^{\rm T}}$, with ${{\bf{S}}_2} \subsetneq {{\bar{\bf{S}}}_1}$.

(4) The steady-state probability staying in a given subset $\mathop {\lim }\limits_{t \to \infty } {p_{{{\bf{S}}_0}}}(t)$

The definition of steady-state probability staying in a given subset is given by

(3.12)\begin{equation} \mathop {\lim }\limits_{t \to \infty } {p_{{{\bf{S}}_0}}}(t) = \mathop {\lim }\limits_{t \to \infty } P\{X(t) \in {{\bf{S}}_0}\}. \end{equation}

Note: When ${\bf{S}}_1 = {\bf{S}}$, $\mathop {\lim }\limits_{t \to \infty } {p_{{{\bf{S}}_1}{{\bf{S}}_2}}}(t)$ in Eq. (3.9) reduces to $\mathop {\lim }\limits_{t \to \infty } {p_{{{\bf{S}}_0}}}(t)$ in Eq. (3.12).

We have

(3.13)\begin{align} p_{{{\bf{S}}_0}}^*(s) =& {{\bf{\beta }}_0}\sum\limits_{r = 0}^\infty {{{[{\bf{G}}_{{{\bf{S}}_0}{{{\bar{\bf{S}}}}_0}}^*(s){\bf{G}}_{{{{\bar{\bf{S}}}}_0}{{\bf{S}}_0}}^*(s)]}^r}} {\bf{P}}_{{{\bf{S}}_0}{{\bf{S}}_0}}^*(s){{\bf{u}}_0}\notag\\ & + {{{\bar{\bf{\beta}}}}_0}{\bf{G}}_{{{{\bar{\bf{S}}}}_0}{{\bf{S}}_0}}^*(s)\sum\limits_{r = 0}^\infty {{{[{\bf{G}}_{{{\bf{S}}_0}{{{\bar{\bf{S}}}}_0}}^*(s){\bf{G}}_{{{{\bar{\bf{S}}}}_0}{{\bf{S}}_0}}^*(s)]}^r}} {\bf{P}}_{{{\bf{S}}_0}{{\bf{S}}_0}}^*(s){{\bf{u}}_0} \notag \\ =& {{\bf{\beta }}_0}{[{\bf{I}} - {\bf{G}}_{{{\bf{S}}_0}{{{\bar{\bf{S}}}}_0}}^*(s){\bf{G}}_{{{{\bar{\bf{S}}}}_0}{{\bf{S}}_0}}^*(s)]^{- 1}}{\bf{P}}_{{{\bf{S}}_0}{{\bf{S}}_0}}^*(s){{\bf{u}}_0} \notag\\ &+ {{{\bar{\bf{\beta}}}}_0}{\bf{G}}_{{{{\bar{\bf{S}}}}_0}{{\bf{S}}_0}}^*(s){[{\bf{I}} - {\bf{G}}_{{{\bf{S}}_0}{{{\bar{\bf{S}}}}_0}}^*(s){\bf{G}}_{{{{\bar{\bf{S}}}}_0}{{\bf{S}}_0}}^*(s)]^{- 1}}{\bf{P}}_{{{\bf{S}}_0}{{\bf{S}}_0}}^*(s){{\bf{u}}_0}, \end{align}

where the initial probability vectors ${{\bf{\beta }}_0} = ({\alpha _1}, \ldots ,{\alpha _{|{{\bf{S}}_0}|}})$, ${{\bar{\bf{\beta}}}_0} = ({\alpha _{|{{\bf{S}}_0}| + 1}}, \ldots ,{\alpha _{|{\bf{S}}|}})$, and ${{\bf{u}}_0} = (\underbrace {1, \ldots ,1}_{\left| {{{\bf{S}}_0}} \right|})^{\rm T}$.

Note: In some contents, the steady-state probability staying at a given subset $\mathop {\lim }\limits_{t \to \infty } {p_{{{\bf{S}}_0}}}(t)$ is equivalent to the steady-state availability $\mathop {\lim }\limits_{t \to \infty } A(t)$ when two subsets coincide, that is, $\mathop {\lim }\limits_{t \to \infty } {p_{{{\bf{S}}_0}}}(t) = \mathop {\lim }\limits_{t \to \infty } A(t)$ when ${\bf{W}} = {{\bf{S}}_0}$.

Based on the four steady-state measures expressed by Eqs. (3.5), (3.6), (3.8), (3.10), (3.11), and (3.13), we only need to consider essentially two terms, which are

(3.14)\begin{align} {{\bf{T}}_1}(s): =& {{\bf{\alpha }}_{\bf{W}}}{[{\bf{I}} - {\bf{G}}_{{\bf{WF}}}^*(s){\bf{G}}_{{\bf{FW}}}^*(s)]^{- 1}}{(s{\bf{I}} - {{\bf{Q}}_{{\bf{WW}}}})^{- 1}} \notag \\ & + {{\bf{\alpha }}_{\bf{F}}} {\bf{G}}_{{\bf{FW}}}^*(s){[{\bf{I}} - {\bf{G}}_{{\bf{WF}}}^*(s){\bf{G}}_{{\bf{FW}}}^*(s)]^{- 1}}{(s{\bf{I}} - {{\bf{Q}}_{{\bf{WW}}}})^{- 1}}, \end{align}

(3.15)\begin{align} {{\bf{T}}_2}(s): = {{\bf{\beta }}_1}{\bf{G}}_{{\bf{FW}}}^*(s){[{\bf{I}} - {\bf{G}}_{{\bf{WF}}}^*(s){\bf{G}}_{{\bf{FW}}}^*(s)]^{- 1}}{(s{\bf{I}} - {{\bf{Q}}_{{\bf{WW}}}})^{- 1}}. \end{align}

Note: when we replace the subsets ${{\bf{S}}_1}$ and ${{\bf{S}}_2}$, ${{\bf{S}}_1}$ and ${{\bar{\bf{S}}}_1}$, ${{\bf{S}}_0}$ and ${{\bar{\bf{S}}}_0}$ by F and W in the corresponding equations, respectively, then the two terms presented in Eqs. (3.14) and (3.15) are given.

Summarizing the above cases, we need to calculate the following two limits: $\mathop {\lim }\limits_{s \downarrow 0} [s{{\bf{T}}_i}(s)]$ $ (i = 1,2)$ for getting the four steady-state measures under the case that s approaches to zero from the right side, which is equivalent to the case that time t approaches to infinity.

4. Proofs on the limiting results

As mentioned above, the four limits do not depend on the initial state probability vector of the underlying Markov process considered, which is a well-known result in Markov processes. However, when using the results in aggregated stochastic process, it is not directly known that the four limits are not relevant to the initial state probability vector. In the following, we present a detailed proof on this conclusion. Before giving the main results, two Lemmas are needed first.

Lemma 4.1. Given a matrix ${{\bf{Q}}_{n \times n}}$, if ${\rm{Rank}}({{\bf{Q}}_{n \times n}}) = n - k$ $(k \in \{1,2, \ldots ,n - 1\} )$, then there exists a nonsingular matrix C such that

(4.1)\begin{equation} {{\bf{Q}}_{n \times n}} = {{\bf{C}}^{- 1}}\left( \begin{array}{c|c} {\bf{0}} &{\bf{B}} \\ \hline {\bf{0}} & {{{\bf{P}}_{(n - k) \times (n - k)}}} \end{array} \right){\bf{C}}, \end{equation}

where ${\rm{Rank}}({{\bf{P}}_{(n - k) \times (n - k)}}) = n - k$.

Corollary 4.1. Let matrix ${\bf{P}}(s): = s{\bf{I}} - {\bf{Q}}$ and ${\rm{Rank}}({\bf{Q}}) = n - k$ $(k \in \{1,2, \ldots ,n - 1\} )$, where Q is the transition rate matrix of Markov process $\{X(t),t \ge 0\} $ with state space ${\bf{S}} = \{1,2, \ldots ,n\} $, then

(4.2)\begin{equation} \det [{\bf{P}}(s)] = {s^k}{P_{n - k}}(k), \end{equation}

where ${P_{n - k}}(s)$ is a polynomial of s with degree n − k and ${P_{n - k}}(0) \ne 0$.

Proof. Based on Lemma 4.1, we have

\begin{equation*}\begin{array}{rlrlrlrlrlrl}\operatorname{det}\lbrack\bf P(s)\rbrack&=\operatorname{det}\left[{s\bf I-\bf C^{-1}\left(\begin{array}{c|c}\mathbf0&\bf B\\\hline\mathbf0&{\bf P}_{(n-k)\times(n-k)}\end{array}\right)\bf C}\right]\\[12pt] &=\operatorname{det}\left[{\bf C^{-1}\left(s\bf I-{\left(\begin{array}{c|c}\mathbf0&\bf B\\\hline\mathbf0&{\bf P}_{(n-k)\times(n-k)}\end{array}\right)}\right)\bf C}\right]\end{array}\end{equation*}

\begin{equation*}\begin{array}{rlrlrlrlrlrl}&=\operatorname{det}{\left[ s\bf I-{\left(\begin{array}{c|c}\mathbf0&\bf B\\\hline\mathbf0&{\bf P}_{(n-k)\times(n-k)}\end{array}\right)}\right]}\\[12pt]& =\operatorname{det}{\left[{\left(\begin{array}{ccc|c}s&&\mathbf0&\\&\ddots&&-\bf B\\\mathbf0&&s&\\\hline&\mathbf0&&s\bf I-{\bf P}_{(n-k)\times(n-k)}\end{array}\right)}\right]}=s^kP_{n-k}(s).\end{array}\end{equation*}

On the other hand, ${\rm{Rank}}( - {{\bf{P}}_{(n - k) \times (n - k)}}(0)) = n - k,$ that is, ${P_{n - k}}(0) \ne 0$, which completes the proof.

Lemma 4.2. For the Markov process $\{X(t),t \ge 0\} $ with state space ${\bf{S}} = {\bf{W}} \cup {\bf{F}}$ and ${\bf{W}} \cap {\bf{F}} = \phi $ and transition rate matrix ${\bf{Q}} = \left( {\matrix{{{{\bf{Q}}_{{\bf{WW}}}}} & {{{\bf{Q}}_{{\bf{WF}}}}} \cr {{{\bf{Q}}_{{\bf{FW}}}}} & {{{\bf{Q}}_{{\bf{FF}}}}} \cr } } \right)$, then

(4.3)\begin{equation} \det [{{\bf{Q}}_{{\bf{WW}}}}({\bf{I}} - {{\bf{G}}_{{\bf{WF}}}}{{\bf{G}}_{{\bf{FW}}}})] = 0. \end{equation}

Proof. We have

\begin{align*} & {{\bf{Q}}_{{\bf{WW}}}}({\bf{I}} - {{\bf{G}}_{{\bf{WF}}}}{{\bf{G}}_{{\bf{FW}}}}){{\bf{u}}_{\bf{W}}}\\ & = [ {{\bf{Q}}_{{\bf{WW}}}} - {{\bf{Q}}_{{\bf{WW}}}}( - {\bf{Q}}_{{\bf{WW}}}^{- 1}{{\bf{Q}}_{{\bf{WF}}}})( - {\bf{Q}}_{{\bf{FF}}}^{- 1}{{\bf{Q}}_{{\bf{FW}}}})]{{\bf{u}}_{\bf{W}}} \\ & = [ {{\bf{Q}}_{{\bf{WW}}}} - {{\bf{Q}}_{{\bf{WF}}}}{\bf{Q}}_{{\bf{FF}}}^{- 1}{{\bf{Q}}_{{\bf{FW}}}})]{{\bf{u}}_{\bf{W}}} \\ & = {{\bf{Q}}_{{\bf{WW}}}}{{\bf{u}}_{\bf{W}}} - {{\bf{Q}}_{{\bf{WF}}}}{\bf{Q}}_{{\bf{FF}}}^{- 1}{{\bf{Q}}_{{\bf{FW}}}}{{\bf{u}}_{\bf{W}}} \\ & = {{\bf{Q}}_{{\bf{WW}}}}{{\bf{u}}_{\bf{W}}} - {{\bf{Q}}_{{\bf{WF}}}}{\bf{Q}}_{{\bf{FF}}}^{- 1}( - {{\bf{Q}}_{{\bf{FF}}}}{{\bf{u}}_{\bf{F}}}) \\ & = {{\bf{Q}}_{{\bf{WW}}}}{{\bf{u}}_{\bf{W}}} + {{\bf{Q}}_{{\bf{WF}}}}{{\bf{u}}_{\bf{F}}} = {\bf{0}}, \end{align*}

that is, the sum of each row of matrix ${{\bf{Q}}_{{\bf{WW}}}}({\bf{I}} - {{\bf{G}}_{{\bf{WF}}}}{{\bf{G}}_{{\bf{FW}}}})$ is zero, then it completes the proof.

Proof. First we have

\begin{align*} & \mathop {\lim }\limits_{s \downarrow 0} {{\bf{T}}_1}(s) = {{\bf{T}}_1}(0) \\ & = {{\bf{\alpha }}_{\bf{W}}} {[{\bf{I}}{\rm{- }}{{\bf{G}}_{{\bf{WF}}}}{{\bf{G}}_{{\bf{FW}}}}]^{- 1}}{( - {{\bf{Q}}_{{\bf{WW}}}})^{- 1}} + {{\bf{\alpha }}_F}{\kern 1pt} {{\bf{G}}_{{\bf{FW}}}}{[{\bf{I}} - {{\bf{G}}_{{\bf{WF}}}}{{\bf{G}}_{{\bf{FW}}}}]^{- 1}}{( - {{\bf{Q}}_{{\bf{WW}}}})^{- 1}} \\ & = - ({{\bf{\alpha }}_{\bf{W}}} + {{\bf{\alpha }}_F}{\kern 1pt} {{\bf{G}}_{{\bf{FW}}}}){\kern 1pt} {[{{\bf{Q}}_{{\bf{WW}}}}({\bf{I}}{\rm{- }}{{\bf{G}}_{{\bf{WF}}}}{{\bf{G}}_{{\bf{FW}}}})]^{- 1}}, \end{align*}

and $\mathop {\lim }\limits_{s \downarrow 0} {{\bf{T}}_2}(s) = - {\bf{\beta }}_1 {{\bf{G}}_{{\bf{FW}}}} {[{{\bf{Q}}_{{\bf{WW}}}}({\bf{I}}{\rm{- }}{{\bf{G}}_{{\bf{WF}}}}{{\bf{G}}_{{\bf{FW}}}})]^{- 1}}$. But from Lemma 4.2, we can know that both $\mathop {\lim }\limits_{s \downarrow 0} {{\bf{T}}_1}(s)$ and $\mathop {\lim }\limits_{s \downarrow 0} {{\bf{T}}_2}(s)$ do not exist. Thus, we cannot directly get the $\mathop {\lim }\limits_{s \downarrow 0} [s{{\bf{T}}_i}(s)]$ $(i = 1,2)$ by replacing s with zero.

Now we consider the inversed matrix ${\bf{P}}(s)$ directly for a given transition rate matrix Q when ${\rm{Rank}}({\bf{Q}}) = n - 1$,

(4.4)\begin{equation} {{\bf{P}}^{- 1}}(s) = {(s{\bf{I}} - {\bf{Q}})^{- 1}} = {1 \over {\det {\bf{P}}(s)}}{{\bf{P}}^*}(s) = {1 \over {s{P_{n - 1}}(s)}}{{\bf{P}}^*}(s), \end{equation}

where ${P_{n - 1}}(s)$ is a polynomial of s such that ${P_{n - 1}}(0) \ne 0$, and the adjugate matrix ${{\bf{P}}^*}(s)$ given by

\begin{equation*} {{\bf{P}}^*}(s) = \left( \begin{matrix} {\det [{{\bf{P}}_{11}}(s)]} & {- \det [{{\bf{P}}_{21}}(s)]} & \cdots & {{{( - 1)}^{n + 1}}\det [{{\bf{P}}_{n1}}(s)]} \\ {- \det [{{\bf{P}}_{12}}(s)]} & {\det [{{\bf{P}}_{22}}(s)]} & \cdots & {{{( - 1)}^{n + 2}}\det [{{\bf{P}}_{n2}}(s)]} \\ \vdots & \vdots & \ddots & \vdots \\ {{{( - 1)}^{n + 1}}\det [{{\bf{P}}_{1n}}(s)]} & {{{( - 1)}^{n + 2}}\det [{{\bf{P}}_{2n}}(s)]} & \cdots & {\det [{{\bf{P}}_{nn}}(s)]} \end{matrix} \right). \end{equation*}

On the other hand, we have ${{\bf{P}}_{ij}}(s) = s{\bf{I}} - {{\bf{Q}}_{ij}},{\rm{~}}i,j \in {\bf{S}},$ where ${{\bf{Q}}_{ij}}$ is a matrix obtained by deleting the ith row and jth column of Q. Obviously, we known that $\det[ {{\bf{P}}_{ij}}(s)]$ is a polynomial of s with degree n − 1. Let $\det [{{\bf{P}}_{ij}}(s)] = P_{n - 1}^{(i,j)}(s)$. Thus, we have

\begin{equation*}\bf P^{-1}(s)=\frac1{sP_{n-1}(s)}{\left( \begin{matrix} P_{n-1}^{(1,1)}(s)& -P_{n-1}^{(2,1)}(s)& \cdots & {(-1)}^{n+1}P_{n-1}^{(n,1)}(s)\\ -P_{n-1}^{(1,2)}(s)& P_{n-1}^{(2,2)}(s)& \cdots& {(-1)}^{n+2}P_{n-1}^{(n,2)}(s)\\ \vdots&\vdots&\ddots&\vdots\\ {(-1)}^{n+1}P_{n-1}^{(1,n)}(s)& {(-1)}^{n+2}P_{n-1}^{(2,n)}(s)& \cdots& P_{n-1}^{(n,n)}(s)\end{matrix}\right)}.\end{equation*}

Besides, we can prove that ${( - 1)^{i + j}}\det [{{\bf{P}}_{ij}}(0)] = {( - 1)^{i + l}}\det [{{\bf{P}}_{il}}(0)]$ for any $i,j,l \in {\bf{S}}$. This is because, if we denote ${p_{ij}}(0) \equiv {p_{ij}},$ for any $i,j \in {\bf{S}},$ then it is clear that ${\bf{P}}(0) = {\left( {{p_{ij}}(0)} \right)_{n \times n}} = - {\bf{Q}}: = {\left( {{p_{ij}}} \right)_{n \times n}}$. Without loss of generality, it is assumed j < l, and then

\begin{align*}&\operatorname{det}\lbrack{\bf P}_{ij}(0)\rbrack\\ &=\operatorname{det}\lbrack{\left(\begin{array}{@{\kern-0.1pt}cccccccccc@{\kern-0.1pt}}p_{11}& \cdots & p_{1(j-1)} & p_{1(j+1)} & \cdots & p_{1(l-1)} & p_{1l} & p_{1(l+1)} & \cdots & p_{1n}\\ \vdots & \ddots & \vdots & \vdots & \ddots & \vdots & \vdots & \vdots & \ddots & \vdots\\ p_{(i-1)1} & \cdots & p_{(i-1)(j-1)} & p_{(i-1)(j+1)} & \cdots & p_{(i-1)(l-1)} & p_{(i-1)l} & p_{(i-1)(l+1)} & \cdots & p_{(i-1)n}\\ p_{(i+1)1} & \cdots & p_{(i+1)(j-1)} & p_{(i+1)(j+1)} & \cdots & p_{(i+1)(l-1)} & p_{(i+1)l} & p_{(i+1)(l+1)} & \cdots & p_{(i+1)n}\\\vdots & \ddots & \vdots & \vdots & \ddots & \vdots & \vdots & \vdots & \ddots & \vdots\\ p_{n1} & \cdots & p_{n(j-1)} & p_{n(j+1)} & \cdots & p_{n(l-1)} & p_{nl} & p_{n(l+1)} & \cdots & p_{nn}\end{array}\right)}\rbrack\\ & =\operatorname{det}\lbrack{\left(\begin{array}{@{\kern-0.1pt}cccccccccc@{\kern-0.1pt}}p_{11} & \cdots & p_{1(j-1)} & p_{1(j+1)} & \cdots & p_{1(l-1)} &- p_{1j} & p_{1(l+1)} & \cdots & p_{1n}\\ \vdots & \ddots & \vdots & \vdots & \ddots & \vdots & \vdots & \vdots & \ddots & \vdots\\ p_{(i-1)1} & \cdots & p_{(i-1)(j-1)} & p_{(i-1)(j+1)} & \cdots & p_{(i-1)(l-1)} & -p_{(i-1)j} & p_{(i-1)(l+1)} & \cdots & p_{(i-1)n}\\ p_{(i+1)1} & \cdots & p_{(i+1)(j-1)} & p_{(i+1)(j+1)} & \cdots & p_{(i+1)(l-1)} & -p_{(i+1)j} & p_{(i+1)(l+1)} & \cdots & p_{(i+1)n}\\ \vdots & \ddots & \vdots & \vdots & \ddots & \vdots & \vdots & \vdots & \ddots & \vdots\\ p_{n1} & \cdots & p_{n(j-1)} & p_{n(j+1)} & \cdots & p_{n(l-1)} & -p_{nj} & p_{n(l+1)} & \cdots & p_{nn}\end{array}\right)}\rbrack.\end{align*}

On the other hand, we have

\begin{align*}\operatorname{det}&\lbrack{\bf P}_{il}(0)\rbrack\\=&\operatorname{det}\lbrack{\left(\begin{array}{@{\kern-0.1pt}cccccccccc@{\kern-0.1pt}}p_{11}&\cdots&p_{1(j-1)}&p_{1j}&p_{1(j+1)}&\cdots&p_{1(l-1)}&p_{1(l+1)}&\cdots&p_{1n}\\\vdots&\ddots&\vdots&\vdots&\vdots&\ddots&\vdots&\vdots&\ddots&\vdots\\p_{(i-1)1}&\cdots&p_{(i-1)(j-1)}&p_{(i-1)j}&p_{(i-1)(j+1)}&\cdots&p_{(i-1)(l-1)}&p_{(i-1)(l+1)}&\cdots&p_{(i-1)n}\\p_{(i+1)1}&\cdots&p_{(i+1)(j-1)}&p_{(i+1)j}&p_{(i+1)(j+1)}&\cdots&p_{(i+1)(l-1)}&p_{(i+1)(l+1)}&\cdots&p_{(i+1)n}\\\vdots&\ddots&\vdots&\vdots&\vdots&\ddots&\vdots&\vdots&\ddots&\vdots\\p_{n1}&\cdots&p_{n(j-1)}&p_{nj}&p_{n(j+1)}&\cdots&p_{n(l-1)}&p_{n(l+1)}&\cdots&p_{nn}\end{array}\right)}\rbrack\\=&\operatorname{det}\lbrack{\left(\begin{array}{@{\kern-0.1pt}cccccccccc@{\kern-0.1pt}}p_{11}&\cdots&p_{1(j-1)}&p_{1(j+1)}&\cdots&p_{1(l-1)}&-p_{1j}&p_{1(l+1)}&\cdots&p_{1n}\\\vdots&\ddots&\vdots&\vdots&\ddots&\vdots&\vdots&\vdots&\ddots&\vdots\\p_{(i-1)1}&\cdots&p_{(i-1)(j-1)}&p_{(i-1)(j+1)}&\cdots&p_{(i-1)(l-1)}&-p_{(i-1)j}&p_{(i-1)(l+1)}&\cdots&p_{(i-1)n}\\p_{(i+1)1}&\cdots&p_{(i+1)(j-1)}&p_{(i+1)(j+1)}&\cdots&p_{(i+1)(l-1)}&-p_{(i+1)j}&p_{(i+1)(l+1)}&\cdots&p_{(i+1)n}\\\vdots&\ddots&\vdots&\vdots&\ddots&\vdots&\vdots&\vdots&\ddots&\vdots\\p_{n1}&\cdots&p_{n(j-1)}&p_{n(j+1)}&\cdots&p_{n(l-1)}&-p_{nj}&p_{n(l+1)}&\cdots&p_{nn}\end{array}\right)}\rbrack\\&\times{(-1)^{(l-1)-j+1}}\\=&{(-1)^{l-j}}\operatorname{det}\lbrack{\bf P}_{ij}(0)\rbrack.\end{align*}

Thus, we have proved that $\det ({{\bf{P}}_{i1}}(0)) = -\det ({{\bf{P}}_{i2}}(0)) = \cdots = {( - 1)^{n - 1}}\det ({{\bf{P}}_{in}}(0))$, for any $i \in {\bf{S}}$, which implies that matrix ${{\bf{P}}^*}(0)$ has the same row, that is, each column in matrix ${{\bf{P}}^*}(0)$ consists in the same value.

Furthermore, based on the result of inversion of partitioned matrix presented in Section 2, we have

\begin{equation*}{(s\bf I-\bf Q)^{-1}}={\left(\begin{array}{@{\kern-0.1pt}cc@{\kern-0.1pt}}{\lbrack\bf P(s)\rbrack}_{\bf W\bf W}&{\lbrack\bf P(s)\rbrack}_{\bf W\bf F}\\{\lbrack\bf P(s)\rbrack}_{\bf F\bf W}&{\lbrack\bf P(s)\rbrack}_{\bf F\bf F}\end{array}\right)},\end{equation*}

where ${[{\bf{P}}(s)]_{{\bf{WW}}}} = {[{\bf{I}} - {\bf{G}}_{{\bf{WF}}}^*(s){\bf{G}}_{{\bf{FW}}}^*(s)]^{- 1}}{\bf{P}}_{{\bf{WW}}}^*(s),$ and

\begin{equation*}{[{\bf{P}}(s)]_{{\bf{FW}}}} = {\bf{G}}_{{\bf{FW}}}^*(s){[{\bf{I}} - {\bf{G}}_{{\bf{WF}}}^*(s){\bf{G}}_{{\bf{FW}}}^*(s)]^{- 1}}{\bf{P}}_{{\bf{WW}}}^*(s).\end{equation*}

On the other hand, from Eq. (4.4), we have

\begin{equation*}{\lbrack\bf P(s)\rbrack_{\bf W\bf W}}=\frac1{sP_{n-1}(s)}{\left(\begin{array}{@{\kern-0.1pt}ccc@{\kern-0.1pt}}P_{n-1}^{(1,1)}(s)&\cdots&{(-1)}^{\vert\bf W\vert+1}P_{n-1}^{(\vert\bf W\vert,1)}(s)\\\vdots&\ddots&\vdots\\{(-1)}^{\vert\bf W\vert+1}P_{n-1}^{(1,\vert\bf W\vert)}(s)&\cdots&{(-1)}^{2\vert\bf W\vert}P_{n-1}^{(\vert\bf W\vert,\vert\bf W\vert)}(s)\end{array}\right)},\end{equation*}

\begin{equation*}{\lbrack\bf P(s)\rbrack_{\bf F\bf W}}=\frac1{sP_{n-1}(s)}{\left(\begin{array}{@{\kern-0.1pt}ccc@{\kern-0.1pt}}{(-1)}^{\vert\bf W\vert+2}P_{n-1}^{(1,\vert\bf W\vert+1)}(s)&\cdots&{(-1)}^{2\vert\bf W\vert+1}P_{n-1}^{(\vert\bf W\vert,\vert\bf W\vert+1)}(s)\\\vdots&\ddots&\vdots\\{(-1)}^{\vert\bf W\vert+\vert\bf F\vert+1}P_{n-1}^{(1,\vert\bf W\vert+\vert\bf F\vert)}(s)&\cdots&{(-1)}^{2\vert\bf W\vert+\vert\bf F\vert}P_{n-1}^{(\vert\bf W\vert,\vert\bf W\vert+\vert\bf F\vert)}(s)\end{array}\right)}.\end{equation*}

Thus, we have

(4.5)\begin{equation} \mathop {\lim }\limits_{s \downarrow 0} [s{(s{\bf{I}} - {\bf{Q}})^{- 1}}] = {1 \over {{P_{n - 1}}(0)}}{{\bf{P}}^*}(0), \end{equation}

with the same row, that is,

\begin{equation*} \begin{array}{*{20}{@{\kern-0.1pt}llll@{\kern-0.1pt}}} {\mathop {\lim }\limits_{s \downarrow 0} [s{{\mathbf{T}}_1}(s)]} \\ { = {{\mathbf{\alpha }}_W}{{\left( {\begin{array}{*{20}{@{\kern-0.1pt}cccc@{\kern-0.1pt}}} {d_{11}^{({\mathbf{W}})}}&{d_{12}^{({\mathbf{W}})}}& \cdots &{d_{1|{\mathbf{W}}|}^{({\mathbf{W}})}} \\ {d_{11}^{({\mathbf{W}})}}&{d_{12}^{({\mathbf{W}})}}& \cdots &{d_{1|{\mathbf{W}}|}^{({\mathbf{W}})}} \\ \vdots & \vdots & \ddots & \vdots \\ {d_{11}^{({\mathbf{W}})}}&{d_{12}^{({\mathbf{W}})}}& \cdots &{d_{1|{\mathbf{W}}|}^{({\mathbf{W}})}} \end{array}} \right)}_{|{\mathbf{W}}| \times |{\mathbf{W}}|}} + {{\mathbf{\alpha }}_{\mathbf{F}}}{{\left( {\begin{array}{*{20}{@{\kern-0.1pt}cccc@{\kern-0.1pt}}} {d_{11}^{({\mathbf{W}})}}&{d_{12}^{({\mathbf{W}})}}& \cdots &{d_{1|{\mathbf{W}}|}^{({\mathbf{W}})}} \\ {d_{11}^{({\mathbf{W}})}}&{d_{12}^{({\mathbf{W}})}}& \cdots &{d_{1|{\mathbf{W}}|}^{({\mathbf{W}})}} \\ \vdots & \vdots & \ddots & \vdots \\ {d_{11}^{({\mathbf{W}})}}&{d_{12}^{({\mathbf{W}})}}& \cdots &{d_{1|{\mathbf{W}}|}^{({\mathbf{W}})}} \end{array}} \right)}_{|{\mathbf{F}}| \times |{\mathbf{W}}|}}} \\ [18pt] { = \left( {\begin{array}{*{20}{@{\kern-0.1pt}cccc@{\kern-0.1pt}}} {d_{11}^{({\mathbf{W}})}[\sum\limits_{i \in {\mathbf{W}}} {{\alpha _i}} + \sum\limits_{i \in {\mathbf{F}}} {{\alpha _i}],} }&{d_{12}^{({\mathbf{W}})}[\sum\limits_{i \in {\mathbf{W}}} {{\alpha _i}} + \sum\limits_{i \in {\mathbf{F}}} {{\alpha _i}],} }&{ \ldots ,}&{d_{1|{\mathbf{W}}|}^{({\mathbf{W}})}[\sum\limits_{i \in {\mathbf{W}}} {{\alpha _i}} + \sum\limits_{i \in {\mathbf{F}}} {{\alpha _i}]} } \end{array}} \right)} \\ [12pt] { = \left( {\begin{array}{*{20}{@{\kern-0.1pt}cccc@{\kern-0.1pt}}} {d_{11}^{({\mathbf{W}})},}&{d_{12}^{({\mathbf{W}})},}&{ \ldots ,}&{d_{1|{\mathbf{W}}|}^{({\mathbf{W}})}} \end{array}} \right),} \end{array} \end{equation*}

where $d_{ij}^{({\bf{W}})} = {( - 1)^{i + j}}{{P_{n - 1}^{(j,i)}(0)} \over {{P_{n - 1}}(0)}}$, $i,j \in {\bf{S}}$. Thus, we can have that $\mathop {\lim }\limits_{s \downarrow 0} [s{{\bf{T}}_1}(s)]$ does not depend on the initial probability vector ${\bf{\alpha }_0} = ({{\bf{\alpha }}_{\bf{W}}},{{\bf{\alpha }}_{\bf{F}}}).$

Similarly, we have that $\mathop {\lim }\limits_{s \downarrow 0} [s{{\bf{T}}_2}(s)]$ does not depend on ${{\bf{\beta }}_1}$. Thus, it completes the proof.

Note: For Theorem 4.1, it seems we can understand that the $\mathop {\lim }\limits_{t \to \infty } A(t)$, $\mathop {\lim }\limits_{t \to \infty } A([t,t + a])$, $\mathop {\lim }\limits_{t \to \infty } {p_{{{\mathbf{S}}_1}{{\mathbf{S}}_2}}}(t)$, and $\mathop {\lim }\limits_{t \to \infty } {p_{{{\bf{S}}_0}}}(t)$ are independent of the initial probability condition for aggregated Markov processes when ${\rm{Rank}}({\bf{Q}}) = n - 1$, namely there is at most one absorbing state for the underlying Markov process, see Examples 5.1 and 5.2, for example. In fact, the limiting behaviors of a finite irreducible aggregated Markov process are independent of the initial probability condition, so that it is also true under condition ${\rm{Rank}}({\bf{Q}}) = n - 1$.

5. Numerical examples and discussion

In this section, some examples will be presented by using the direct computation way to illustrate the independence of initial probability vectors for direct limits such as steady-state interval and point availabilities, steady-state transition probability between two subsets, and steady-state probability staying in a given subset, which have all been proved and expressed in Theorem 4.1.

Example 5.1. (Case that ${\rm{Rank}}({\bf{Q}}) = n - 1$ and all states communicate.) Suppose that n = 4, all states in ${\bf{S}} = \{1,2,3,4\} $ communicate with each other, with ${\bf{W}} = \{1,2,3\} ,{\bf{F}} = \{4\} $ and

\begin{equation*}\bf Q={\left(\begin{array}{ccc|c}-6&1&4&1\\1&-3&1&1\\1&1&-4&2\\\hline2&1&2&-5\end{array}\right)}.\end{equation*}

The transition diagram of $\{X(t),t \ge 0\} $ is shown in Figure 1.

Figure 1. The transition diagram for Example 5.1.

We have ${\rm{Rank}}({\bf{Q}}) = 3$, and

\begin{equation*}s{(s\bf I-\bf Q)^{-1}}={\left(\begin{array}{ccc|c}\frac{s^3+12s^2+41s+35}{s^3+18s^2+106s+200}&\frac1{s+4}&\frac{4s^2+35s+70}{s^3+18s^2+106s+200}&\frac{s^2+16s+45}{s^3+18s^2+106s+200}\\\frac{s^2+12s+35}{s^3+18s^2+106s+200}&\frac1{s+4}&\frac{(s+7)(s+10)}{s^3+18s^2+106s+200}&\frac{s^2+13s+45}{s^3+18s^2+106s+200}\\\frac{s^2+13s+35}{s^3+18s^2+106s+200}&\frac1{s+4}&\frac{s^3+14s^2+59s+70}{s^3+18s^2+106s+200}&\frac{2s^2+20s+45}{s^3+18s^2+106s+200}\\\hline\frac{2s^2+17s+35}{s^3+18s^2+106s+200}&\frac1{s+4}&\frac{(2s+7)(s+10)}{s^3+18s^2+106s+200}&\frac{s^3+13s^2+48s+45}{s^3+18s^2+106s+200}\end{array}\right)}.\end{equation*}

Taking the limit, we have

\begin{equation*}\underset{s\downarrow 0}{\operatorname{lim}\lbrack}{}s{(s\bf I-\bf Q)^{-1}}\rbrack=\frac1{40}{\left(\begin{array}{ccc|c}7&10&14&9\\7&10&14&9\\7&10&14&9\\\hline7&10&14&9\end{array}\right)},\end{equation*}

which shows that the matrixes $\mathop {\lim }\limits_{s \downarrow 0} [s{{\bf{T}}_1}(s)] ={1 \over {40}} (7, 10,14)$ and $\mathop {\lim }\limits_{s \downarrow 0} [s{{\bf{T}}_2}(s)]={1 \over {40}} (7, 10,14)$ possess the same row; thus, the related steady-state indexes do not depend on the initial probability vector.

Example 5.2. (Case that ${\rm{Rank}}({\bf{Q}}) = n - 1$ but not all states communicate with each other.) Suppose that n = 4, three states in ${\bf{S}} = \{1,2,3,4\} $ communicate with each other, but state 4 does not communicate with states 1, 2, and 3, with ${\bf{W}} = \{1,2,3\} ,{\bf{F}} = \{4\} $ and,

\begin{equation*}\bf Q={\left(\begin{array}{ccc|c}-3&1&2&0\\2&-5&3&0\\3&1&-4&0\\\hline1&2&3&-6\end{array}\right)}.\end{equation*}

The transition diagram of $\{X(t),t \ge 0\} $ is shown in Figure 2.

Figure 2. The transition diagram for Example 5.2.

We have ${\rm{Rank}}({\bf{Q}}) = 3$, and

\begin{equation*}s{(s\bf I-\bf Q)^{-1}}={\left(\begin{array}{ccc|c}\frac{s^2+9s+17}{s^2+12s+36}&\frac1{s+6}&\frac{2s+13}{s^2+12s+36}&0\\\frac{2s+17}{s^2+12s+36}&\frac{s+1}{s+6}&\frac{3s+13}{s^2+12s+36}&0\\\frac{3s+17}{s^2+12s+36}&\frac1{s+6}&\frac{s^2+8s+13}{s^2+12s+36}&0\\\hline\frac{s^2+22s+102}{(s^2+12s+36)(s+6)}&\frac{2(s+3)}{{(s+6)}^2}&\frac{3s^2+32s+78}{(s^2+12s+36)(s+6)}&\frac s{s+6}\end{array}\right)}.\end{equation*}

Taking the limit, we have

\begin{equation*}\underset{s\downarrow 0}{\operatorname{lim}\lbrack}{}s{(s\bf I-\bf Q)^{-1}}\rbrack=\frac1{36}{\left(\begin{array}{ccc|c}17&6&13&0\\17&6&13&0\\17&6&13&0\\\hline17&6&13&0\end{array}\right)},\end{equation*}

which shows that the matrixes $\mathop {\lim }\limits_{s \downarrow 0} [s{{\bf{T}}_1}(s)] ={1 \over {36}} (17, 6,13)$ and $\mathop {\lim }\limits_{s \downarrow 0} [s{{\bf{T}}_2}(s)]={1 \over {36}} (17, 6,13)$ possess the same row; thus, the related steady-state indexes do not depend on the initial probability vector, although all states do not communicate anymore.

Example 5.3. (Case that ${\rm{Rank}}({\bf{Q}}) = n - 2$, some states communicate and some not.) Suppose that n = 6, the state space ${\bf{S}} = \{1,2,3,4,5,6\} $, with ${\bf{W}} = \{1,2,3\} ,{}{\bf{F}} = \{4,5,6\}$ and

\begin{equation*}\bf Q={\left(\begin{array}{ccc|ccc}-2&2&0&0&0&0\\1&-1&0&0&0&0\\1&2&-8&1&1&3\\\hline2&3&1&-10&3&1\\0&0&0&0&-2&2\\0&0&0&0&1&-1\end{array}\right)}.\end{equation*}

Denote ${\bf{S}} = {{\bf{S}}_1} \cup {{\bf{S}}_2} \cup {{\bf{S}}_3}$ with ${{\bf{S}}_1} = \{1,2\} $, ${{\bf{S}}_2} = \{3,4\}$, $ {{\bf{S}}_3} = \{5,6\} $. The states in ${{\bf{S}}_i}$ $ (i = 1,2,3)$ communicate with each other, and ${{\bf{S}}_2} \to {{\bf{S}}_1}, $ ${{\bf{S}}_1}\nrightarrow{{\bf{S}}_2}$ and ${{\bf{S}}_2} \to {{\bf{S}}_3},$ $ {{\bf{S}}_3}\nrightarrow{{\bf{S}}_2}$, that is, some states communicate each other, but some states do not. The transition diagram of $\{X(t),t \ge 0\} $ is shown in Figure 3.

Figure 3. The transition diagram for Example 5.3.

We have ${\rm{Rank}}({\bf{Q}}) = 4$, and

\begin{equation*}s{(s\bf I-\bf Q)^{-1}}=\frac1{s+3}{\left(\begin{array}{ccc|ccc}s+1&2&0&0&0&0\\1&s+2&0&0&0&0\\v_{31}&v_{32}&v_{33}&v_{34}&v_{35}&v_{36}\\\hline v_{41}&v_{42}&v_{43}&v_{44}&v_{45}&v_{46}\\0&0&0&0&s+1&2\\0&0&0&0&1&s+2\end{array}\right)},\end{equation*}

where

\begin{align*} &{v_{31}} = {{{s^2} + 15s + 35} \over {{s^2} + 18s + 79}},\qquad {v_{32}} = {{2{s^2} + 29s + 70} \over {{s^2} + 18s + 79}},\qquad{v_{33}} = {{s(s+10)(s + 3)} \over {{s^2} + 18s + 79}},\\ &{v_{34}} = {{s(s + 3)} \over {{s^2} + 18s + 79}},\qquad{v_{35}} = {{{s^2} + 17s + 44} \over {{s^2} + 18s + 79}},\qquad{v_{36}} = {{3{s^2} + 39s + 88} \over {{s^2} + 18s + 79}},\\ &{v_{41}} = {{2{s^2} + 22s + 43} \over {{s^2} + 18s + 79}},\qquad{v_{42}} = {{3{s^2} + 36s + 86} \over {{s^2} + 18s + 79}},\qquad{v_{43}} = {{s(s + 3)} \over {{s^2} + 18s + 79}},\\ &{v_{44}} = {{s(s + 8)(s + 3)} \over {{s^2} + 18s + 79}},\qquad{v_{45}} = {{3{s^2} + 29s + 36} \over {{s^2} + 18s + 79}},\qquad{v_{46}} = {{{s^2} + 19s + 72} \over {{s^2} + 18s + 79}}. \end{align*}

Taking the limit, we have

\begin{equation*}\mathop {\lim }\limits_{s \downarrow 0} \left[ {s{{\left( {s{\bf{\it{I}}} - {\mathbf{Q}}} \right)}^{ - 1}}} \right] = \frac{1}{3}\left( {\begin{array}{*{20}{ccc|ccc}} 1&2&0&0&0&0 \\ 1&2&0&0&0&0 \\ {\frac{{35}}{{79}}}&{\frac{{70}}{{79}}}&0&0&{\frac{{44}}{{79}}}&{\frac{{88}}{{79}}} \\ \hline {\frac{{43}}{{79}}}&{\frac{{86}}{{79}}}&0&0&{\frac{{36}}{{79}}}&{\frac{{72}}{{79}}} \\ 0&0&0&0&1&2 \\ 0&0&0&0&1&2 \end{array}} \right),\end{equation*}

which tells us that when ${\rm{Rank}}({\bf{Q}}) \le n - 2$, the situation becomes complicated, which is our future research work.

In fact, from the knowledge of common finite irreducible Markov processes, we also have that the initial conditions do not affect the steady-state measures, regardless of the rank of transition rate matrix. Because, in general, for a finite state space of Markov process $\{X(t),t \ge 0\} $, we have $\mathop {\lim }\limits_{t \to \infty } {P_{ij}}(t) = \mathop {\lim }\limits_{t \to \infty } P\{X(t) = j|X(0) = i\} = {\pi _j}$ and $\mathop {\lim }\limits_{t \to \infty } {P_i}(t) = \mathop {\lim }\limits_{t \to \infty } P\{X(t) = i\} = {\pi _i}$, both do not depend on the initial conditions of the underlying Markov process.

For aggregated Markov processes, if the steady-state measures are expressed by the Laplace transforms, it is not easy to known this result holds or not in terms of these Laplace transform formulas. In the previous contents, we prove it under the case of ${\rm{Rank}}({\bf{Q}}) = n - 1$, while for the other cases that ${\rm{Rank}}({\bf{Q}}) \le n - 2$, it is sure that the conclusion holds too, but it seems to be complicated to prove it. Here a numerical example is presented for the case of ${\rm{Rank}}({\bf{Q}}) = n - 2$ in which the effects of the initial probability vector on the steady-state measures can be studied more deeply.

Example 5.3 also seems to provide a way to generate a transition rate matrix with ${\rm{Rank}}({\bf{Q}}) = n - k, $ $k \in \{1,2, \ldots ,n - 1\} $, and gives some guidance to understand how the initial state probability vector affects the steady-state measures. Because, for example, the steady state probabilities staying at subsets ${{\bf{S}}_1}$ and ${{\bf{S}}_2}$ are $\mathop {\lim }\limits_{t \to \infty } {p_{{{\bf{S}}_1}}}(t) ={{\alpha}_1}+{{\alpha}_2}+{35\over {79}}{{\alpha}_3}+{43\over {79}}{{\alpha}_4}$ and $\mathop {\lim }\limits_{t \to \infty } {p_{{{\bf{S}}_3}}}(t) ={44\over {79}}{{\alpha}_3}+{36\over {79}}{{\alpha}_4}+{{\alpha}_5}+{{\alpha}_6}$. It is obvious to know that the steady-state probabilities depend on the initial probability vector ${{\bf{\alpha}}_0}=({{\alpha}_1},{{\alpha}_2},\ldots,{{\alpha}_6})$ through ${{\alpha}_1}+{{\alpha}_2}$ and ${{\alpha}_5}+{{\alpha}_6}$, respectively, instead of individual ${{\alpha}_1}$, ${{\alpha}_2}$ and ${{\alpha}_5},{{\alpha}_6}$.

6. Conclusion

In the paper, under the condition that ${\rm{Rank}}({\bf{Q}}) = n - 1$, we directly prove that four limiting measures expressed by the Laplace transforms derived by using aggregated Markov processes do not depend on the initial state, which are well-known in common Markov processes. The condition ${\rm{Rank}}({\bf{Q}}) = n - 1$, which is more extensive than the case of time reversibility in ion channel modeling, includes the case that all states communicate with each other. Our work implies that (1) four limiting measures do not depend on the initial probability vector ${{\bf{\alpha }}_0}$, which is a well-known result in Markov repairable systems in terms of properties of Markov processes, but now this is a direct way to prove this result in aggregated Markov processes; (2) this proof can bridge a gap between the common knowledge in Markov processes and aggregated stochastic processes for the case of steady-state situation; (3) similar problems can be discussed, which are via a direct way based on the results in aggregated stochastic processes; (4) the initial probability vector ${{\bf{\alpha }}_0}$ appears in the formulas, but these limited results do not depend on this initial vector, which is an extension of one-dimensional case in which the initial value may disappear via the division of the same value in the formulas.

The possible future research work may include the study of similar problems under the cases of aggregated Semi-Markov processes, of ${\rm{Rank}}({\bf{Q}}) \le n - 2$, and of time omission problems, and so on. We believe that our work can solid the theory and applications of aggregated Markov processes on the related limiting measures, especially in reliability field.