As recognized in a large number of studies, the quantitative evaluation of fault-tolerant computer systems needs to deal simultaneously with aspects of both performance and dependability. For this purpose, Meyer [66] developed the concept of performability, which may be interpreted as the cumulative performance over a finite mission time. The increasing need to evaluate cumulative measures comes from the fact that in highly available systems, steady-state measures can be very poor, even if the mission time is not small. Considering, for instance, critical applications, it is crucial for the user to ensure that the probability that the system will achieve a given performance level is high enough.

Formally, the system fault-repair behavior is assumed to be modeled by a homogeneous Markov chain. Its state space is divided into disjoint subsets, which represent the different configurations of the system. A performance level or reward rate is associated with each of these configurations. This reward rate quantifies the ability of the system to perform in the corresponding configuration. Performability is then the accumulated reward over the mission time.

Simulating a Markov model is the method of choice when analytic or numerical approaches are not possible. Basically, the former happens when the model is not regular enough, meaning that it does not have enough structure, the latter when its size precludes the use of numerical procedures. We also use the term Monte Carlo to refer to these simulation methods, using the term in its most common meaning, i.e. the use of randomness to solve deterministic problems. There are many important subfields in this wide area that have been the object of concerted research efforts, for instance the MCMC [83], [31] or the perfect simulation [81] approaches, and many procedures designed for specific application areas, such as queuing [52], [38], [109], financial problems [37], physics [40], and many more [45], [35], [4]. Monte Carlo techniques are extremely powerful, and they can deal, in principle, with any kind of model. They are not limited by the lack of structure in the model, nor by the model's size. However, they have their own difficulties, the main one being the case of rare events. If you want to analyze a specific metric quantifying some aspect of this rarity, the fact that the focused event appears with very small probability means that it will also be hard to observe in the simulation.

In this chapter we analyze the conditions under which the aggregated process constructed from a homogeneous Markov chain over a given partition of its state space is also a homogeneous Markov chain. The results of this chapter mainly come from [89] and [91] for irreducible discrete-time Markov chains, in [94] for irreducible continuous-time Markov chains, and [58] for absorbing Markov chains. A necessary condition to obtain a Markovian aggregated process has been obtained in [58]. These works are based on the pioneering works of [50] and [1]. They are themselves the basis of other extensions such as [56], which deals with infinite state spaces, or [57] in which the author derives new results for the lumpability of reducible Markov chains and obtains spectral properties associated with lumpability.

State aggregation in irreducible DTMC

Introduction and notation

In this first section we consider the aggregated process constructed from an irreducible and homogeneous discrete-time Markov chain (DTMC), X, over a given partition of the state space. We analyze the conditions under which this aggregated process is also Markov homogeneous and we give a characterization of this situation.

The Markov chain X = {Xn, n ∈ N} is supposed to be irreducible and homogeneous. The state space is supposed to be finite and denoted by S = {1, 2, …, N}. All the vectors used are row vectors except when specified. For instance, the column vector with all its entries equal to 1 is denoted by 1 and its dimension is defined by the context.

Preliminary words

From the theoretical point of view, Markov chains are a fundamental class of stochastic processes. They are the most widely used tools for solving problems in a large number of domains. They allow the modeling of all kinds of systems and their analysis allows many aspects of those systems to be quantified. We find them in many subareas of operations research, engineering, computer science, networking, physics, chemistry, biology, economics, finance, and social sciences. The success of Markov chains is essentially due to the simplicity of their use, to the large set of theoretical associated results available, that is, the high degree of understanding of the dynamics of these stochastic processes, and to the power of the available algorithms for the numerical evaluation of a large number of associated metrics.

In simple terms, the Markov property means that given the present state of the process, its past and future are independent. In other words, knowing the present state of the stochastic process, no information about the past can be used to predict the future. This means that the number of parameters that must be taken into account to represent the evolution of a system modeled by such a process can be reduced considerably. Actually, many random systems can be represented by a Markov chain, and certainly most of the ones used in practice. The price to pay for imposing the Markov property on a random system consists of cleverly defining the present of the system or equivalently its state space.

One of the approaches that can be followed when dealing with very large models is to try to compute the bounds of the metrics of interest. This chapter describes bounding procedures that can be used to bound two important dependability metrics. We are concerned with highly dependable systems; the bounding techniques described here are designed to be efficient in that context.

First, we address the situation where the states of the model are weighted by real numbers, the model (the Markov chain) is irreducible and the metric is the asymptotic mean reward. The most important example of a metric of this type in dependability is the asymptotic availability, but many other metrics also fit this framework, for instance, the mean number of customers in a queuing system, or in part of a queuing network (in this case, the techniques must be considered in light traffic conditions, the equivalent to a highly dependable system in this queuing situation).

Second, we consider the evaluation of the Mean Time To Failure, that is, of the expectation of the delay from the initial instant to the occurrence of the first system's failure. The situation is that of a chain where all states but one are transient and the remaining state is absorbing; the MTTF corresponds to the mean absorption time of the chain.