The aim of this paper is to define the entropy of a finite semi-Markov process. We define the entropy of the finite distributions of the process, and obtain explicitly its entropy rate by extending the Shannon–McMillan–Breiman theorem to this class of nonstationary continuous-time processes. The particular cases of pure jump Markov processes and renewal processes are considered. The relative entropy rate between two semi-Markov processes is also defined.

We present a stochastic algorithm which generates optimal probabilities for the chaos game to decompress an image represented by the fixed point of an IFS operator. The algorithm can be seen as a sort of time-inhomogeneous regenerative process. We prove that optimal probabilities exist and, by martingale methods, that the algorithm converges almost surely. The method holds for IFS operators associated with any arbitrary number of possibly overlapping affine contraction maps on the pixels space.

We present an entropy conservation principle applicable to either discrete or continuous variables which provides a useful tool for aggregating observations. The associated method of modality grouping transforms a variable Z1 into a new variable Z2 such that the mutual information I(Z2,Y) between Y, a variable of interest, and Z2 is equal to I(Z1,Y).

We prove a monotonicity property for a function of general square integrable pairs of martingales which is useful in fractal-based algorithms for compression of image data.

Intersymbol and cochannel interference in a communications channel can often be modelled as the sum of an infinite series of random variables with weights which decay fairly rapidly. Frequently, this yields a random variable which is singular but non-atomic. The Hausdorff dimension of the distribution is estimated and methods for calculating expectations are studied. A connection is observed between the dimension and the complexity of the calculation of an expectation.