Skip to main content Accessibility help

We use cookies to distinguish you from other users and to provide you with a better experience on our websites. Close this message to accept cookies or find out how to manage your cookie settings.

Close cookie message
×
  1. Home
  2. Browse subjects
  3. Computer Science
  4. Distributed, Networked and Mobile Computing
  5. Content listing

Refine search

Refine search

Access:
Content type:
Publication date:
Apply   From and To filters
Subject: Show more
Journals Show more
Publishers: Show more
Societies Show more
Series: Show more
Collections: Show more

Actions for selected content:

Select all | Deselect all
  • View selected items
  • Export citations
  • Download PDF (zip)
  • Save to Kindle
  • Save to Dropbox
  • Save to Google Drive

Save Search

You can save your searches here and later view and run them again in "My saved searches".

Please provide a title, maximum of 40 characters.
Cancel
×

1968 results in Distributed, Networked and Mobile Computing

Page 44 of 99

  • 16 - Distributed Transfer Learning via Cooperative Matrix Factorization

  • from Part Three - Alternative Learning Settings
    • By Evan Xiang, Hong Kong University of Science and Technology, Nathan Liu, Hong Kong University of Science and Technology, Qiang Yang, Hong Kong University of Science and Technology
  • Edited by Ron Bekkerman, Mikhail Bilenko, John Langford
  • Book:
    Scaling up Machine Learning
    Published online:
    05 February 2012
    Print publication:
    30 December 2011, pp 331-351

  • Summary

    Machine learning and data-mining technologies have already achieved significant success in many knowledge engineering areas including web search, computational advertising, recommender systems, etc. A major challenge in machine learning is the data sparsity problem. For example, in the domain of online recommender systems, we attempt to recommend information items (e.g., movies, TV, books, news, images, web pages, etc.) that are likely to be of interest to the user. However, the item space is usually very large and the amount of user preference values is small. When the user data are too sparse, it is difficult to obtain a reliable and useful model for recommendation. Whereas large online sites like Amazon and Google can easily access huge volumes of user data, the enormous number of smaller online business sites, which collectively constitute the long tail of the web, are much more likely to have very sparse user data and have difficulty in generating accurate recommendations. One potential solution to the data sparsity problem is to transfer knowledge from other information sources (e.g., Mehta and Hofmann, 2007; Li, Yang, and Xue, 2009). Such techniques for knowledge transfer are called transfer learning (see, e.g., Pan and Yang, 2010). An additional issue is that, in reality, many small websites often attract similar users and/or provide similar items, if not the identical ones, which implies that data about such users/items could potentially be distributed across different systems. For example, Delicious and Digg are both popular online social bookmarking tools.

  • 17 - Interactive Channel Coding

  • from Part III - Multihop Networks
  • Abbas El Gamal, Young-Han Kim, University of California, San Diego
  • Book:
    Network Information Theory
    Published online:
    05 June 2012
    Print publication:
    08 December 2011, pp 427-458

  • Summary

    The network models we studied so far involve only one-way (feedforward) communication. Many communication systems are inherently interactive, allowing for cooperation through feedback and information exchange over multiway channels. In this chapter, we study the role of feedback in communication and present results on the two-way channel introduced by Shannon as the first multiuser channel. The role of multiway interaction in compression and secure communication will be studied in Chapters 20 and 22, respectively.

    As we showed in Section 3.1.1, the capacity of a memoryless point-to-point channel does not increase when noiseless causal feedback is present. Feedback can still benefit point-to-point communication, however, by simplifying coding and improving reliability. The idea is to first send the message uncoded and then to use feedback to iteratively reduce the receiver's error about the message, the error about the error, and so on. We demonstrate this iterative refinement paradigm via the Schalkwijk–Kailath coding scheme for the Gaussian channel and the Horstein and block feedback coding schemes for the binary symmetric channel. We show that the probability of error for the Schalkwijk–Kailath scheme decays double-exponentially in the block length, which is significantly faster than the single-exponential decay of the probability of error without feedback.

    We then show that feedback can enlarge the capacity region in multiuser channels. For the multiple access channel, feedback enlarges the capacity region by enabling statistical cooperation between the transmitters. We show that the capacity of the Gaussian MAC with feedback coincides with the outer bound obtained by allowing arbitrary (instead of product) joint input distributions. For the broadcast channel, feedback can enlarge the capacity region by enabling the sender to simultaneously refine both receivers’ knowledge about the messages. For the relay channel, we show that the cutset bound is achievable when noiseless causal feedback from the receiver to the relay is allowed. This is in contrast to the case without feedback in which the cutset bound is not achievable in general.

    Finally, we discuss the two-way channel, where two nodes wish to exchange their messages interactively over a shared noisy channel. The capacity region of this channel is not known in general. We first establish simple inner and outer bounds on the capacity region.

  • 13 - Multiple Description Coding

  • from Part II - Single-Hop Networks
  • Abbas El Gamal, Young-Han Kim, University of California, San Diego
  • Book:
    Network Information Theory
    Published online:
    05 June 2012
    Print publication:
    08 December 2011, pp 320-335

  • Summary

    We consider the problem of generating two descriptions of a source such that each description by itself can be used to reconstruct the source with some desired distortion and the two descriptions together can be used to reconstruct the source with a lower distortion. This problem is motivated by the need to efficiently communicate multimedia content over networks such as the Internet. Consider the following two scenarios:

  • • Path diversity: Suppose we wish to send a movie to a viewer over a network that suffers from data loss and delays. We can send multiple copies of the same description of the movie to the viewer via different paths in the network. Such replication, however, is inefficient and the viewer does not benefit from receiving more than one copy of the description. Multiple description coding provides a more efficient means to achieve such “path diversity.” We generate multiple descriptions of the movie, so that if the viewer receives only one of them, the movie can be reconstructed with some acceptable quality, and if the viewer receives two of them, the movie can be reconstructed with a higher quality and so on.

  • • Successive refinement: Suppose we wish to send a movie with different levels of quality to different viewers. We can send a separate description of the movie to each viewer. These descriptions, however, are likely to have significant overlaps. Successive refinement, which is a special case of multiple description coding, provides a more efficient way to distribute the movie. The idea is to send the lowest quality description and successive refinements of it (instead of additional full descriptions). Each viewer then uses the lowest quality description and some of the successive refinements to reconstruct the movie at her desired level of quality.

    • The optimal scheme for generating multiple descriptions is not known in general. We present the El Gamal–Cover coding scheme for generating two descriptions that are individually good but still carry additional information about the source when combined together. The proof of achievability uses the multivariate covering lemma in Section 8.4. We show that this scheme is optimal for the quadratic Gaussian case. The key to the converse is the identification of a common-information random variable. We then present an improvement on the El Gamal–Cover scheme by Zhang and Berger that involves sending an additional common description.

  • 8 - General Broadcast Channels

  • from Part II - Single-Hop Networks
  • Abbas El Gamal, Young-Han Kim, University of California, San Diego
  • Book:
    Network Information Theory
    Published online:
    05 June 2012
    Print publication:
    08 December 2011, pp 197-226

  • Summary

    We resume the discussion of broadcast channels started in Chapter 5. Again consider the 2-receiver DM-BCp(y1, y2|x) with private and common messages depicted in Figure 8.1. The definitions of a code, achievability, and capacity regions are the same as in Chapter 5. As mentioned before, the capacity region of the DM-BC is not known in general. In Chapter 5, we presented the superposition coding scheme and showed that it is optimal for several classes of channels in which one receiver is stronger than the other. In this chapter, we study coding schemes that can outperform superposition coding and present the tightest known inner and outer bounds on the capacity region of the general broadcast channel.

    We first show that superposition coding is optimal for the 2-receiver DM-BC with degraded message sets, that is, when either R1 = 0 or R2 = 0. We then show that superposition coding is not optimal for BCs with more than two receivers. In particular, we establish the capacity region of the 3-receiver multilevel BC. The achievability proof involves the new idea of indirect decoding, whereby a receiver who wishes to recover only the common message still uses satellite codewords in decoding for the cloud center.

    We then present Marton's inner bound on the private-message capacity region of the 2-receiver DM-BC and show that it is optimal for the class of semideterministic BCs. The coding scheme involves the multicoding technique introduced in Chapter 7 and the new idea of joint typicality codebook generation to construct dependent codewords for independent messages without the use of a superposition structure. The proof of the inner bound uses the mutual covering lemma, which is a generalization of the covering lemma in Section 3.7. Marton's coding scheme is then combined with superposition coding to establish an inner bound on the capacity region of the DM-BC that is tight for all classes of DM-BCs with known capacity regions. Next, we establish the Nair–El Gamal outer bound on the capacity region of the DM-BC. We show through an example that there is a gap between these inner and outer bounds. Finally, we discuss extensions of the aforementioned coding techniques to broadcast channels with more than two receivers and with arbitrary messaging requirements.

  • 22 - Information Theoretic Secrecy

  • from Part IV - Extensions
  • Abbas El Gamal, Young-Han Kim, University of California, San Diego
  • Book:
    Network Information Theory
    Published online:
    05 June 2012
    Print publication:
    08 December 2011, pp 549-582

  • Summary

    Confidentiality of information is a key consideration in many networking applications, including e-commerce, online banking, and intelligence operations. How can information be communicated reliably to the legitimate users, while keeping it secret from eavesdroppers? How does such a secrecy constraint on communication affect the limits on information flow in the network?

    In this chapter, we study these questions under the information theoretic notion of secrecy, which requires each eavesdropper to obtain essentially no information about the messages sent from knowledge of its received sequence, the channel statistics, and the codebooks used. We investigate two approaches to achieve secure communication. The first is to exploit the statistics of the channel from the sender to the legitimate receivers and the eavesdroppers. We introduce the wiretap channel as a 2-receiver broadcast channel with a legitimate receiver and an eavesdropper, and establish its secrecy capacity, which is the highest achievable secret communication rate. The idea is to design the encoder so that the channel from the sender to the receiver becomes effectively stronger than the channel to the eavesdropper; hence the receiver can recover the message but the eavesdropper cannot. This wiretap coding scheme involves multicoding and randomized encoding.

    If the channel from the sender to the receiver is weaker than that to the eavesdropper, however, secret communication at a positive rate is not possible. This brings us to the second approach to achieve secret communication, which is to use a secret key shared between the sender and the receiver but unknown to the eavesdropper. We show that the rate of such secret key must be at least as high as the rate of the confidential message. This raises the question of how the sender and the receiver can agree on such a long secret key in the first place. After all, if they had a confidential channel with sufficiently high capacity to communicate the key, then why not use it to communicate the message itself!

    We show that if the sender and the receiver have access to correlated sources (e.g., through a satellite beaming common randomness to them), then they can still agree on a secret key even when the channel has zero secrecy capacity. We first consider the source model for key agreement, where the sender communicates with the receiver over a noiseless public broadcast channel to generate a secret key from their correlated sources.

  • 21 - Communication for Computing

  • from Part IV - Extensions
  • Abbas El Gamal, Young-Han Kim, University of California, San Diego
  • Book:
    Network Information Theory
    Published online:
    05 June 2012
    Print publication:
    08 December 2011, pp 529-548

  • Summary

    In the first three parts of the book we investigated the limits on information flow in networks whose task is to communicate (or store) distributed information. In many real world distributed systems, such as multiprocessors, peer-to-peer networks, networked mobile agents, and sensor networks, the task of the network is to compute a function, make a decision, or coordinate an action based on distributed information. Can the communication rate needed to perform such a task at some node be reduced relative to communicating all the sources to this node?

    This question has been formulated and studied in computer science under communication complexity and gossip algorithms, in control and optimization under distributed consensus, and in information theory under coding for computing and the μ -sum problem, among other topics. In this chapter, we study information theoretic models for distributed computing over networks. In some cases, we find that the total communication rate can be significantly reduced when the task of the network is to compute a function of the sources rather than to communicate the sources themselves, while in other cases, no such reduction is possible.

    We first show that the Wyner–Ziv theorem in Chapter 11 extends naturally to the case when the decoder wishes to compute a function of the source and the side information. We provide a refined characterization of the lossless special case of this result in terms of conditional graph entropy. We then discuss distributed coding for computing. Although the rate–distortion region for this case is not known in general (even when the goal is to reconstruct the sources themselves), we show through examples that the total communication rate needed for computing can be significantly lower than for communicating the sources themselves. The first example we discuss is the μ -sum problem, where the decoder wishes to reconstruct a weighted sum of two separately encoded Gaussian sources with a prescribed quadratic distortion. We establish the rate–distortion region for this setting by reducing the problem to the CEO problem discussed in Chapter 12. The second example is lossless computing of the modulo-2 sum of a DSBS. Surprisingly, we find that using the same linear code at both encoders can outperform Slepian–Wolf coding.

Page 44 of 99