If servers were free, then every server farm would have an infinite number of servers, and no job would ever have to wait in a queue. Unfortunately, servers are not free to buy, and they are also not free to operate. Running servers consumes lots of power, and even leaving a server on, but idle, still consumes nearly 60% of the power consumed by a busy server [15]. Given these costs, it pays to spend some time thinking about how many servers one really needs. This subject is called capacity provisioning.

Observe that we can actually already, in theory, answer the question of how many servers we need to achieve a given Quality of Service (QoS) goal, based on the formulas we derived in Chapter 14. In that chapter, we considered the M/M/k server farm model and derived expressions for the distribution of the number of jobs in the system, the probability of queueing, PQ, and the expected response time, E[T]. Given a QoS constraint on E[T] or PQ, we can iterate over these formulas and deduce the exact number of servers, k, needed to achieve the desired constraint.

However, iterating over a formula is time consuming and also does not provide any intuitions for the result. The purpose of this chapter is to formulate intuitions and rules of thumb for understanding how many servers we need to achieve a certain QoS goal, and what the impact is of increasing the number of servers.