A1.1 Basic matrix operations

A1.1.1 Definition (matrix)

A matrix is an ordered rectangular array of numbers (or functions). The numbers (or functions) are called the elements or the entries of the matrix.

A1.1.2 Dimension/size of a matrix

A matrix having m rows and a columns is a matrix of dimension m x n.

CHANCE PERMEATES OUR physical and mental universe. While the role of chance in human lives has had a longer history, starting with the more authoritative influence of the nobility, the more rationally sound theory of probability and statistics has come into practice in diverse areas of science and engineering starting from the early to mid-twentieth century. Practical applications of statistical theories proliferated to such an extent in the previous century that the American government-sponsored RAND corporation published a 600-page book that wholly consisted of a random number table and a table of standard normal deviates. One of the primary objectives of this book was to enable a computer-simulated approximate solution of an exact but unsolvable problem by a procedure known as the Monte Carlo method devised by Fermi, von Neumann, and Ulam in the 1930s–40s.

Statistical methods are the mainstay of conducting modern scientific experiments. One such experimental paradigm is known as a randomized control trial, which is widely used in a variety of fields such as psychology, drug verification, testing the efficacy of vaccines, agricultural sciences, and demography. These statistical experiments require sophisticated sampling techniques in order to nullify experimental biases. With the explosion of information in the modern era, the need to develop advanced and accurate predictive capabilities has grown manifold. This has led to the emergence of modern artificial intelligence (AI) technologies. Further, climate change has become a reality of modern civilization. Accurate prediction of weather and climatic patterns relies on sophisticated AI and statistical techniques. It is impossible to think of a modern economy and social life without the influence and role of chance, and hence without the influence of technological interventions based on statistical principles. We must begin this journey by learning the foundational tenets of probability and statistics.

EMPIRICAL TECHNIQUES rely on abstracting meaning from observable phenomena by constructing relationships between different observations. This process of abstraction is facilitated by appropriate measurements (experiments), suitable organization of data generated by measurements, and, finally, rigorous analysis of the data. The latter is a functional exercise that synthesizes information (data) and theory (model) and enables prediction of hitherto unobserved phenomena.1 It is important to underscore that a good theory (model) that explains a certain phenomenon well by appealing to a set of laws and conditions is expected to be a good candidate for predicting the same using reliable data. For example, a good model for the weight of a normal human being is w = m * h, where w and h refer to weight and height of the person, and m can be set to unity if appropriate units are chosen. A rational explanation of such a formula for weight based on anatomical considerations is perhaps very reasonable. From an empirical standpoint, if we collect height and weight data of normal humans, we will notice that a linear model of the form w = m * h represents the data reasonably well and may be used to predict the weight of the person based on the height of the person. This fact ascertains a functional symmetry between explanation and prediction. Therefore, a good predictive model must automatically be able to explain the data (and related events) well.

A2.1 Mathematical operations

Try the following operations and commands on the command line in the command window of Matlab.

A2.1.1 Defining an array and arranging in ascending and descending order

QUEUES FEATURE IN our daily lives like never before. From the checkout counter in the community grocery store to customer support over the phone, queues are theatres of great social and engineering drama. Entire business operations of many leading companies are geared towards providing hassle-free customer support and experience – timely and effective resolution of client queries about services on a regular basis. Alternatively, it could be effective traffic management and resource optimization for a multiplex cinema operator involved in ticket sales. Sometimes it may not involve humans at all, like in the case of a database query to a computer server for specific information that may be routed through a job queue. How a queue moves in time and how services are offered over epochs determine how businesses will be able to make profit or how efficiently computer servers will execute tasks. All these have a huge technological and economical impact. No wonder we have seen huge investments by concerned stakeholders to upgrade and upscale hardware and software infrastructure to re-engineer queues towards greater system efficiency and profitability. The mathematical technology of queues is crafted out of models that investigate and replicate stochastic behavior of engineering systems. This is the subject of our study in this chapter.

STATISICAL EXPERIMENTS ENABLE us to make inferences from data about parameters that characterize a population. Generally speaking, inferences may be of two types, namely, deductive inference and inductive inference. Deductive inference pertains to conclusions based on a set of premises (propositions) and their synthesis. Deductive reasoning has a definitive character. For example, all men are mortal (first proposition); Socrates is a man (second proposition); hence, Socrates is mortal (deductive conclusion). On the other hand, inductive inference has a probabilistic character. One conducts an experiment and collects data. Based on this data, certain conclusions are drawn that may have a broader applicability beyond the contours of the particular experiment performed by the researcher. This generalization of the conclusions drawn from the particular experiment constitutes the framework of inductive reasoning. For example, measurement of heights of a small group of people belonging to a certain population is conducted. Based on the calculations of this small sample set, and upon finding that for this small group the average height of men is greater than the average height of women, it is inferred that the men of this population are generally taller than the women.

The formal practice of inductive reasoning dates back to the thesis of Gottfried Wilhelm Leibniz (see Figure 5.1). He was the first to propose that probability is a relation between hypothesis and evidence (data). His thesis was founded on three conceptual pillars: chance (probability), possibilities (realizable random events), and ideas (generalization of inferences by induction). We have encountered the first two concepts in earlier chapters of this textbook. In this chapter, we will delve into the third theme whereby we will discuss methods to draw conclusions from data derived from statistical experiments based on the principles of inductive reasoning.

Our lived experiences are punctuated by events that are sometimes a result of our purposeful intentions and at other times outcomes that happen by pure chance. Even at an abstract level, it is a very human endeavor to deduce meaning from seemingly random observations an exercise whose primary objective is to derive a causal structure in observed phenomena. In fact, our whole intellectual pursuit that differentiates us from other beings can be understood through our inner urge to discover the very purpose of our existence and the conditions that make this possible. This eternal play between chance episodes and purposeful volition manifests in diverse situations that I have labored to recreate through computer simulations of realistic events. This play has a dual role - first, it binds together the flow of our varied experiences and, second, it offers us a perspective to assimilate our understanding of events happening around us that affect us. In order to appreciate this play of chance and purpose, it is essential that students and readers have a conceptual grounding in the areas of probability, statistics, and stochastic processes. Therefore, several playful computer simulations and projects are interlaced with theoretical foundations and numerical examples - both solved and exercise problems. In this way, the presentation in this book remains true to its spirit of inviting thoughtful readers to the various aspects of this area of study.

Historical remark

The advent of a rigorous framework for studying probability and statistics dates back to the eighth century AD and is documented in the works of Al-Khalil, who was an Arab philologist. This branch of mathematics continues to be under development with major contributions from Soviet mathematician Andrey N. Kolmogorov, who developed the modern foundations of probability and statistical theory from a measure-theoretic standpoint in the twentieth century.

DISTRIBUTIONS ARE GENERALIZATIONS of mathematical functions from a purely technical standpoint. But perhaps it is most pertinent to begin by asking a more utilitarian question. Why should we study distributions? Specifically, why should we study probability distributions? One of the motivations stems from a practical limitation of experimental measurements that is underlined by the uncertainty principle postulated by Werner Heisenberg (see Figure 2.1). The very fabric of reality and the structure of scientific laws that govern our ability to understand physical phenomena demand a probabilistic (statistical) approach. Our inability to make infinite-precision measurements of data necessitates the consideration of averages over many measurements, and under similar conditions, as a more reliable strategy to affix experimental values to unknowns with reasonable accuracy.

The advent of the internet and sensor technology has enabled humankind to collect, store, and share data in bulk. In turn, access to a variety of data has amplified a different kind of problem, which is to devise an appropriate strategy to derive meaning from data. Indeed, extracting information from data has acquired the highest priority among tasks performed by engineers and scientists alike. State-ofthe-art machine learning algorithms are used to process and analyze data in order to leverage maximum gains in developing new technology and creating a new body of knowledge.

Further, the data-rich tech-universe has inherent complexity in addition to the vastness in terms of numbers. This complexity arises from the fact that often this data is embedded in a higher-dimensional space. For example, the data acquired by a camera hosted on a robot is in the form of multiple grayscale images (frames); each data-frame is constituted of a sequence of numbers that represents the intensity of grayness of each pixel. If each image has a resolution 100 × 100 (pixel count), then this image data is embedded in a 10000 dimensional space. Additionally, if the camera records 100 frames per second for one minute, then we have 6000 data points in a 10000 dimensional space. This is just an illustrative example of how a high-dimensional large data set may be generated. Quite evidently, not all the 10000 dimensions host most of the information. One of the most important techniques that we will learn in this chapter will allow us to extract a lower dimensional representation of the data set that will retain sufficient information for the robot to navigate and perform its tasks.

MARKOV CHAINS WERE first formulated as a stochastic model1 by Russian mathematician Andrei Andreevich Markov. Markov spent most of his professional career at St. Petersburg University and the Imperial Academy of Science. During this time, he specialized in the theory of numbers, mathematical analysis, and probability theory. His work on Markov chains utilized finite square matrices (stochastic matrices) to show that the two classical results of probability theory, namely, the weak law of large numbers and the central limit theorem, can be extended to the case of sums of dependent random variables. Markov chains have wide scientific and engineering applications in statistical mechanics, financial engineering, weather modeling, artificial intelligence, and so on. In this chapter, we will look at a few applications as we build the concepts of Markov chains. Additionally, we will also implement a technique (using Markov chains) to solve a simple and practical engineering problem related to aircraft control and automation.

3.1 Chapter objectives

The chapter objectives are listed as follows.

• 1. Students will learn the definition and applications of Markov processes.

• 2. Students will learn the definition of the stochastic matrix (also known as the probability transition matrix) and perform simple matrix calculations to compute conditional probabilities.

• 3. Students will learn to solve engineering and scientific problems based on discrete time Markov chains (DTMCs) using multi-step transition probabilities.

• 4. Students will learn to compute return times and hitting times to Markov states.

• 5. Students will learn to classify different Markov states.

• 6. Students will learn to use the techniques of DTMCs introduced in this chapter to solve a complex engineering problem related to flight control operations.