You pick up your iPhone while waiting in line at a coffee shop. You Google a not-so-famous actor and get linked to a Wikipedia entry listing his recent movies and popular YouTube clips. You check out user reviews on IMDb and pick one, download that movie on BitTorrent or stream it in Netflix. But for some reason the WiFi logo on your phone is gone and you're on 3G. Video quality starts to degrade a little, but you don't know whether it's the video server getting crowded in the cloud or the Internet is congested somewhere. In any case, it costs you $10 per gigabyte, and you decide to stop watching the movie, and instead multitask between sending tweets and calling your friend on Skype, while songs stream from iCloud to your phone. You're happy with the call quality, but get a little irritated when you see that you have no new followers on Twitter.

You've got a typical networked life, an online networked life.

And you might wonder how all these technologies “kind of” work, and why sometimes they don't. Just flip through the table of contents of this book. It's a mixture: some of these questions have well-defined formulations and clear answers while for others there is still a significant gap between the theoretical models and actual practice; a few don't even have widely accepted problem statements. This book is about formulating and answering these 20 questions.

By the end of this chapter, you will count yourself lucky to get as much as a few percent of the advertised speed. Where did the rest go?

A Short Answer

First of all, the terms 3G and 4G can be confusing. There is one track following the standardization body 3GPP called UMTS or WCDMA, and another track in 3GPP2 called CDMA2000. Each also has several versions inbetween 2G and 3G, often called 2.5G, such as EDGE, EVDO, etc. For 4G, the main track is called Long Term Evolution (LTE), with variants such as LTE light and LTE advanced. Another competing track is called WiMAX. Some refer to evolved versions of 3G, such as HSPA+, as 4G too. All these have created quite a bit of confusion in a consumer's mind as to what really is a 3G technology and what really is a 4G technology.

You might have read that the 3G downlink speed for stationary users should be 7.2 Mbps. But when you try to download an email attachment of 3 MB, it often takes as long as one and half minutes. You get around 267 kbps, 3.7% of what you might expect. Who took away the 96%?

Many countries are moving towards LTE. They use a range of techniques to increase the spectral efficiency, defined as the number of bits per second that each Hz of bandwidth can support. These include methods like OFDM and MIMO mentioned at the end of the last chapter and splitting a large cell into smaller ones.

A Short Answer

Almost all of our utility bills are based on the amount we consume: water, electricity, gas, etc. But even though wireless cellular capacity is expensive to provide and difficult to crank up, consumers in some countries like the USA have been enjoying flat-rate buffets for mobile Internet access for many years. Can a restaurant keep offering buffets with the same price if its customers keep doubling their appetites every year? Or will it have to stop at some point?

In April 2010, AT&T announced its usage-based pricing for 3G data users. This was followed in March 2011 by Verizon Wireless for its iPhone and iPad users, and in June 2011 for all of its 3G data users. In July 2011, AT&T started charging fixed broadband users on U-Verse services on the basis of usage too. In March 2012, AT&T announced that those existing customers on unlimited cellular data plans will see their connection speeds throttled significantly once the usage exceeds 5 GB, effectively ending the unlimited data plan. The LTE data plans from both AT&T and Verizon Wireless for the “new iPad” launched soon after no longer offered any type of unlimited data options. In June 2012, Verizon Wireless updated their cellular pricing plans. A customer could have unlimited voice and text in exchange for turning an unlimited data plan to usage-based. AT&T followed with a similar move one month later. What a reversal going from limited voice and unlimited data to unlimited voice and limited data. Similar measures have been pursued, or are being considered, in many other countries around the world for 3G, 4G, and even wired broadband networks.

We saw in Chapter 13 that the Internet provides a “best effort,” i.e., “no effort” service. So, how can it support video distribution that often imposes stringent demands on throughput and delay?

A Short Answer

Viewing models

Watching video is a significant part of many people's daily life, and it is increasingly dependent on the Internet and wireless networks. Movies, TV shows, and home videos flow from the cloud through the IP network to mobile devices. This trend is changing both the networking and the entertainment industries. As of 2011, there were more than 100 million IPTV users in the USA, and Youtube and Netflix together take up about half of the Internet capacity usage. As the trend of decoupling among contents, content delivery channels, and content-consuming devices intensifies, IP has become the basis of almost all the content distribution systems.

This trend is bringing about a revolution in our viewing habits.

1. • Content type: Both user-generated and licensed content have become prevalent. Clearly, more user-generated content implies an increasing need for upload capacity, which is traditionally designed to be much smaller than download capacity.

2. • When: For many types of video content, we can watch them anytime we want, with the help of devices like a Digital Video Recorder (DVR) on IPTV or services like HBO Go.

3. • Where: We can watch video content almost anywhere, at least anywhere with a sufficiently fast Internet connection.

4. […]

We just went through some of the key concepts behind the TCP/IP thin waist of the Internet protocol stack. We will now go through five more chapters on technology networks, focusing on two major trends: massive amounts of content distribution and the prevalent adoption of mobile wireless technologies.

Scaling up the distribution of content, including video content, can be carried out either through the help of peers or by using large data centers. These two approaches, P2P and cloud, are described in this chapter and the next, respectively. In particular, P2P illustrates a key principle behind the success of the Internet: under-specify protocols governing the operation of a network so that an overlay network can be readily built on top of it for future applications unforeseen by today's experts. It also illustrates the importance of backward compatibility, incremental deployability, and incentive alignment in the evolution of the Internet.

A Short Answer

Skype allows phone calls between IP-based devices (like laptops, tablets, and smartphones) or between IP devices and normal phones. It is free for IP-to-IP calls. How could that be? Part of the answer is that it uses a peer-to-peer (P2P) protocol riding on top of IP networks.

P2P started becoming popular around 1999. For example, Kazaa and Gnutella were widely used P2P file- and music-sharing systems back then. However, incentives were not properly designed in those first-generation P2P systems; there were a lot of free riders who did not contribute nearly as much as they consumed.

A Short Answer

Take a look at your iPhone, Android phone, or a smartphone running on some other operating system. It embodies a remarkable story of technology innovations. The rise of wireless networks, the Internet, and the web over the last five decades, coupled with advances in chip design, touchscreen material, battery packaging, software systems, business models… led to this amazing device you are holding in your hand. It symbolizes our age of networked life.

These phones have become the mobile, lightweight, smart centers of focus in our lives. They are used not just for voice calls, but also for data applications: texting, emailing, browsing the web, streaming videos, downloading books, uploading photos, playing games, or video-conferencing friends. The throughputs of these applications are measured in bits per second (bps). These data fly through a cellular network and the Internet. The cellular network in turn consists of the radio air-interface and the core network. We focus on the air-interface part in this chapter, and turn to the cellular core network in Chapter 19.

Terrestrial wireless communication started back in the 1940s, and cellular networks have gone through generations of evolution since the 1970s, moving into what we hear as 4G these days. Back in the 1980s, some estimated that there would be 1 million cellular users in the USA by 2000. That turned out to be one of those way-off under-estimates that did not even get close to the actual impact of networking technologies.

We have come to the last chapter, on a sensitive subject that we touched upon many times in the previous chapters and forms an essential part of both social choice theory and technology network design: quantifying fairness of resource allocation. This may sound obvious, but it does not hurt to highlight: the scope of our discussion will be only on performance metrics, not on liberty and rights.

A Short Answer

Thinking about fairness

The naive view of “equality is fairness” is problematic in examining performance metrics of a group of users stemming from some allocation of resources. If you have to choose from an allocation of (1, 1) Mbps between two iPad users, and an allocation of (100, 101) Mbps, many people would choose (100, 101) Mbps even though it deviates from an equal allocation. Magnitude matters. Part of Rawls' theory of justice is the difference principle that we will discuss in the Advanced Material, which prefers a less equal allocation if that means everyone gets more. Of course, a more challenging choice would have been between (1, 1) Mbps and (1, 2) Mbps.

Another objection to marking equal allocations as the most fair stems from the differences in the contributions by, and the needs of, different users. If a user in a social network glues the entire network together, her contribution is higher than that of a “leaf node” user. If one works twice as hard or twice as effectively as another, these two people should not receive identical salaries.

In this and the next three chapters, we will walk through a remarkable landscape of intellectual foundations. But sometimes we will also see significant gaps between theory and practice.

A Short Answer

We continue with the theme of recommendation. Webpage ranking in Chapter 3 turns a graph into a rank-ordered list of nodes. Movie ranking in Chapter 4 turns a weighted bipartite user–movie graph into a set of rank-ordered lists of movies, with one list per user. We now examine the aggregation of a vector of rating scores by reviewers of a product or service, and turn that vector into a scalar for each product. These scalars may in turn be used to rank order a set of similar products. In Chapter 6, we will further study aggregation of many vectors into a single vector.

When you shop on Amazon, likely you will pay attention to the number of stars shown below each product. But you should also care about the number of reviews behind that averaged number of stars. Intuitively, you know that a product with two reviews, both 5 stars, might not be better than a competing product with one hundred reviews and an average of 4.5 stars, especially if these one hundred reviews are all 4 and 5 stars and the reviewers are somewhat trustworthy. We will see how such intuition can be sharpened.

In most online review systems, each review consists of three fields:

1. rating, a numerical score often on the scale of 1–5 stars (this is the focus of our study),

2. review, in the form of text, and

3. review of review, often a binary up or down vote.

In the last chapter, we saw the importance of topology to functionality. In this and the next chapters, we will focus on generative models of network topology and reverse-engineering of network functionality. These are mathematical constructions that try to explain widespread empirical observations about social and technological networks: the “small world” property and the “scale free” property. We will also highlight common misunderstandings and misuse of generative models.

A Short Answer

Since Milgram's 1967 experiment, the small world phenomenon, or the six degrees of separation, has become one of the most widely told stories in popular science books. Milgram asked 296 people living in Omaha, Nebraska to participate in the experiment. He gave each of them a passport-looking letter, and the destination was in a suburb of Boston, Massachusetts, with the recipient's name, address, and occupation (stockbroker) shown. Name and address sound obvious, and it turned out that it was very helpful to know the occupation. The goal was to send this letter to one of your friends, defined as someone you knew by first name. If you did not know the recipient by first name, you had to send the letter via others, starting with sending it to a friend (one hop), who then sent it to one of her friends (another hop), until the letter finally arrived at someone who knew the recipient by first name and sent it to the recipient. This is illustrated in Figure 9.1.

A Short Answer

Principles of distributed congestion control

When demand exceeds supply, we have congestion. If the supply is fixed, we must reduce the demand to alleviate congestion. Suppose the demand comes from different nodes in a network, we need to coordinate it in a distributed way.

As the demand for capacity in the Internet exceeds the supply every now and then, congestion control becomes essential. The timescale of congestion control is on the order of ms, in contrast to shaping consumer behavior through pricing in Chapters 11 and 12. The need for congestion control was realized in October 1986, when the Internet had its first congestion collapse. It took place over a short, three-hop connection between Lawrence Berkeley Lab and UC Berkeley. The normal throughput was 32 kbps (that is right, kbps, not the Mbps numbers we hear these days). That kind of dial-up modem speed was already low enough, but during the congestion event, it dropped all the way down to 40 bps, by almost a factor of 1000.

The main reason was clear as we saw from the last chapter on routing: when users send so many bits per second that their collective load on a link exceeds the capacity of that link, these packets are stored in a buffer and they wait in the queue to be transmitted. But when that wait becomes too long, more incoming packets accumulate in the buffer until the buffer overflows and packets get dropped.

A Short Answer

ISPs charging consumers on the basis of usage is just one corner of the overall landscape of Internet economics. We will pick consumers' monthly bills to focus on in this chapter, but there are many other key questions.

• The formation of the Internet is driven in part by economic considerations. Different ISPs form peering and transit relationships that are based on business and political decisions as much as on technical ones.

• The invention, adoption, and failure of Internet technologies are driven by the economics of vendor competition and consumer adoption.

• The investment of network infrastructure, from purchasing wireless licensed spectrum to deploying triple-play broadband access, is driven by the economics of capital expenditure, operational expenditure, and returns on investment.

The economics of the Internet are interesting because the technology-economics interactions are bidirectional: economic forces shape the evolution of technology, while disruptive technologies can rewrite the balance of economic equations. This field is also challenging to study because of the lack of publicly available data on ISPs' cost structures and the difficulty of collecting well calibrated consumer data.

Smart data pricing

There is a rapidly growing research field and industry practice on network access pricing. What we described on usage pricing in the last chapter, in the form of tiered and then metered/throttled plans, is just a starter.

We have mentioned the Internet many times so far, and all the previous chapters rely on its existence. It is about time to get into the architecture of the Internet, starting with these two chapters on the TCP/IP foundation of the Internet.

A Short Answer

We will be walking through several core concepts behind the evolution of the Internet, providing the foundation for the next four chapters. So the “short answer” section is going to be longer than the “long answer” section in this chapter.

It is tricky to discuss the historical evolution of technologies like the Internet. Some of what we would like to believe to be the inevitable results from careful design are actually the historical legacy of accidents, or the messy requirements of backward compatibility, incremental deployability, and economic incentives. It is therefore not easy to argue about what could have happened, what could have been alternative paths in the evolution, and what different tradeoffs might have been generated.

Packet switching

The answer to this chapter's question starts with a fundamental idea in designing a network: when your typical users do not really require a dedicated resource, you should allow users to share resources. The word “user” here is used interchangeably with “session.” The logical unit is an application session rather than a physical user or device. For now, assume a session has just one source and one destination, i.e., a unicast session.

A Short Answer

Much of the web services and online information is “free” today because of the advertisements shown on the websites. It is estimated that the online ad industry worldwide reached $94.2 billion in 2012. Compared with traditional media, online advertisements' revenue ranked right below TV and above newspapers.

In the early days of the web, i.e., 1994–1995, online advertisements were sold as banners on a per-thousand-impression basis. But seeing an ad does not mean clicking on it or buying the advertised product or service afterwards. In 1997, GoTo (which later became Overture) started selling advertisement spaces on a per-click basis. This middle ground between ad revenue (what the website cares about) and effectiveness of ad (what the advertisers care about) became a commonly accepted foundation for online advertising.

With the rise of Google came one of the most stable online ad market segments: search ads, also called sponsored search. In 2002, Google started the AdWords service where you can create your ad, attach keywords to it, and send it to Google's database. When someone searches for a keyword, Google will return a list of search results, as well as a list of ads on the right panel, or even the main panel, if that keyword matches any of the keywords of ads in its database. This process takes place continuously and each advertiser can adjust her bids frequently. There are often many ad auctions happening at the same time too. We will skip these important factors in the basic models in this chapter, focusing just on a single auction.

A Short Answer

Now we turn to the other links you see on a search-result webpage; not the ads or sponsored search results, but the actual ranking of webpages by search engines such as Google. We will see that, each time you search on www.google.com, Google solves a very big system of linear equation to rank the webpages.

The idea of embedding links in text dates back to the middle of the last century. As the Internet scaled up, and with the introduction of the web in 1989, the browser in 1990, and the web portal in 1994, this vision was realized on an unprecedented scale. The network of webpages is huge: somewhere between 40 billion and 60 billion according to various estimates. And most of them are connected to each other in a giant component of this network. It is also sparse: most webpages have only a few hyperlinks pointing inward from other webpages or pointing outward to other webpages. Google search organizes this huge and sparse network by ranking the webpages.

More important webpages should be ranked higher. But how do you quantify how important a webpage is? Well, if there are many other important webpages pointing towards webpage A, A is probably important. This argument implicitly assumes two ideas:

1. • Webpages form a network, where a webpage is a node, and a hyperlink is a directed link in the network: webpage A may point to webpage B without B pointing back to A.

2. We can turn the seemingly circular logic of “important webpages pointing to you means you are important” into a set of equations that characterize the equilibrium (a fixed-point equilibrium, not a game-theoretic Nash equilibrium) in terms of a recursive definition of “importance.” This importance score will then act as an approximation of the ultimate test of search engines: how useful a user finds the search results.