As networks continue to grow rapidly in size and complexity, it has become increasingly clear that their evolution is closely tied to a detailed understanding of network traffic. Large IP networks are designed with the goal of providing high availability and low delay/loss while keeping operational complexity and cost low. Meeting these goals is a highly challenging task and can only be achieved through a detailed knowledge of the network and its dynamics.

No matter how surprising this may seem, IP network management today is primarily reactive in nature and relies on trial and error when problems arise. Network operators have limited visibility into the traffic that flows on top of their network, the operational state of the network elements and the behavior of the protocols responsible for the routing of traffic and the reliable transmission of packets from end to end. Furthermore, design and planning decisions only partially rely on actual usage patterns. There are a few reasons behind such a phenomenon.

First, the designers of IP networks have traditionally attached less importance to network monitoring and resource accounting than to issues such as distributed management, robustness to failures and support for diverse services and protocols. Thus, IP network elements (routers and end hosts) have not been designed to retain detailed information about the traffic flowing through them, and IP protocols typically do not provide detailed information about the state of the underlying network.