The world of intelligent and interactive systems is undergoing an era of unprecedented innovation and advanced development. With the rapid progress in natural sensing and perceptual computing technologies, devices and machines are increasingly being endowed with the abilities to sense and understand the world, navigate in the environment, and interact with humans in natural ways. Interfaces based on touch sensing and speech recognition are now ubiquitous, and the race is on to the next frontiers of machine intelligence and interactions based on three-dimensional (3D) sensing. In this paper, we review the recent progress in the development of real-time 3D-sensing technologies and their deployment in a new class of interactive and autonomous systems. As an example of a commercially available platform, we describe the Intel® RealSense technologies, as well as its deployments in a new class of interactive and autonomous systems and applications.

We are currently living in a world dominated by mobile apps and connected devices. State-of-the-art mobile phones and tablets use apps to organize knowledge and information, control devices, and/or complete transactions via local, web, and cloud services. However, users are challenged to select a suite of apps, from the millions available today, that is right for them. Apps are increasingly differentiated only by the user experience and a few specialized functions; therefore, many apps are needed in order to cover all of the services a specific user needs, and the user is often required to frequently switch between apps to achieve a specific goal. User experience is further limited by the inability of apps to effectively interoperate, since relevant user data are often wholly contained within the app. This limitation significantly undermines the continuous (function) flow across apps to achieve a desired goal. The result is a disjointed user experience requiring app switching and replicating data among apps. With these limitations in mind, it appears as if the current mobile experience is nearing its full potential but failing to leverage the full power of modern mobile devices. In this paper, we present a vision of the future where apps are no longer the dominant customer interaction in the mobile world. The alternative that we propose would “orchestrate” the mobile experience by using a “moment-first” model that would leverage machine learning and data mining to bridge a user's needs across app boundaries, matching context, and knowledge of the user with ideal services and interaction models between the user and device. In this way, apps would be employed at a function level, while the overall user experience would be optimized, by liberating user data outside of the app container and intelligently orchestrating the user experience, to fulfill the needs of the moment. We introduce the concept of a functional entry-point and apply the simple label “FUNN” to it (which was named “FUNC” in (Wang, 2014)). We further discuss how a number of learning models could be utilized in building this relationship between the user, FUNN, and context to enable search, recommendations and presentation of FUNNs through a multi-modal human–machine interface that would better fulfill users' needs. Two examples are showcased to demonstrate how this vision is being implemented in home entertainment and driving scenarios. In conclusion, we envision moving forward into a FUNN-based mobile world with a much more intelligent user experience model. This in turn would offer the opportunity for new relationships and business models between software developers, OS providers, and device manufacturers.

With Version 2 of the high-efficiency video coding standard, a new compression efficiency tool targeting redundancies among color components is specified for all 4:4:4 profiles, and referred to as cross-component prediction (CCP). This paper describes and analyses two additional extensions to the specified CCP variant. In the first extension, an additional predictor is introduced. Particularly, beside the luma component, also the first chroma component can serve as a reference for prediction of the second chroma component. The second extension proposes a method for predicting the CCP model parameter from the statistics of already reconstructed neighboring blocks. A performance analysis of coding RGB content in different color representations is given in comparison with CCP and both extensions. Experimental results show that the proposed extensions can improve the compression efficiency effectively compared with CCP, when applied in the YCbCr domain.

This discussion of responsive media provides a perspective on the future of media experiences that are increasingly responsive to users' preference, alertness and their physical, digital, and social environment. By examining a range of future scenarios combining virtual-, remote-, and augmented-reality, autonomous vehicles, digital assistants and robots, we see that the responsiveness of media is what provides the key value. To reach the ultimate goal of augmented innovation in which thinking machines supplement humans, there are a number of technological and user-experience challenges that the research community needs to resolve. These challenges fall into a few key categories: throughput, latency, perception, intelligence, and interaction. While some challenges may be tackled purely technologically, others require insights from sociology and psychology to break new ground. The paper concludes that intelligent, responsive media will not fully supplant human intelligence, but will increasingly serve as augmentation to human creativity.

A free-viewpoint image can be synthesized using color and depth maps of reference viewpoints, via depth-image-based rendering (DIBR). In this process, three-dimensional (3D) warping is generally used. A 3D warped image consists of disocclusion holes with missing pixels that correspond to occluded regions in the reference images, and non-disocclusion holes due to limited sampling density of the reference images. The non-disocclusion holes are those among scattered pixels of a same region or object. These holes are larger when the reference viewpoints and the free viewpoint images have a larger physical distance. Filling these holes has a crucial impact on the quality of free-viewpoint image. In this paper, we focus on free-viewpoint image synthesis that is precisely capable of filling the non-disocclusion holes caused by limited sampling density, using superpixel segmentation. In this approach, we proposed two criteria for segmenting depth and color data of each reference viewpoint. By these criteria, we can detect which neighboring pixels should be connected or kept isolated in each references image, before being warped. Polygons enclosed by the connected pixels, i.e. superpixel, are inpainted by k-means interpolation. Our superpixel approach has a high accuracy since we use both color and depth data to detect superpixels at the location of the reference viewpoint. Therefore, once a reference image that consists of superpixels is 3D warped to a virtual viewpoint, the non-disocclusion holes are significantly reduced. Experimental results verify the advantage of our approach and demonstrate high quality of synthesized image when the virtual viewpoint is physically far from the reference viewpoints.

Spectrum scarcity has driven enhancements of Long-Term Evolution (LTE) in utilizing unlicensed bands in conjunction with licensed bands for delivering mobile data, resulting in the introduction of LTE unlicensed technologies such as Rel-13 LTE–Licensed-Assisted Access (LAA), Rel-14 LTE–Enhanced Licensed-Assisted Access (eLAA), and LTE-Unlicensed (LTE-U). The next-generation radio access technology, 5G New Radio(NR), faces greater technical challenge due to the need to support frequency bands covering various spectrum licensing regimes and a wide range of frequencies (up to 100 GHz) with very different signal propagation characteristics. This paper presents an overview of LAA and eLAA technical features and 5G NR design considerations to achieve a unified access in licensed and unlicensed bands.

This paper proposes a new voice activity detection (VAD) algorithm based on statistical noise suppression and framewise speech/non-speech classification. Although many VAD algorithms have been developed that are robust in noisy environments, the most successful ones are related to statistical noise suppression in some way. Accordingly, we formulate our VAD algorithm as a combination of noise suppression and subsequent framewise classification. The noise suppression part is improved by introducing the idea that any unreliable frequency component should be removed, and the decision can be made by the remaining signal. This augmentation can be realized using a few additional parameters embedded in the gain-estimation process. The framewise classification part can be either model-less or model-based. A model-less classifier has the advantage that it can be applied to any situation, even if no training data are available. In contrast, a model-based classifier (e.g., neural network-based classifier) requires training data but tends to be more accurate. The accuracy of the proposed algorithm is evaluated using the CENSREC-1-C public framework and confirmed to be superior to many existing algorithms.

We highlight the need for – and describe initial strategies to find – new digital-optical basis functions and performance merit functions to serve as a foundation for designing, analyzing, characterizing, testing, and comparing a range of computational imaging systems. Such functions will provide a firm theoretical foundation for computational sensing and imaging and enhanced design software, thereby broadly speeding the development of computational imaging systems.

Artificial Intelligence (AI) was the inspiration that shaped computing as we know it today. In this article, I explore why and how AI would continue to inspire computing and reinvent it when Moore's Law is running out of steam. At the dawn of computing, Alan Turing proposed that instead of comprising many different specific machines, the computing machinery for AI should be a Universal Digital Computer, modeled after human computers, which carry out calculations with pencil on paper. Based on the belief that a digital computer would be significantly faster, more diligent and patient than a human, he anticipated that AI would be advanced as software. In modern terminology, a universal computer would be designed to understand a language known as an Instruction Set Architecture (ISA), and software would be translated into the ISA. Since then, universal computers have become exponentially faster and more energy efficient through Moore's Law, while software has grown more sophisticated. Even though software has not yet made a machine think, it has been changing how we live fundamentally. The computing revolution started when the software was decoupled from the computing machinery. Since the slowdown of Moore's Law in 2005, the universal computer is no longer improving exponentially in terms of speed and energy efficiency. It has to carry ISA legacy, and cannot be aggressively modified to save energy. Turing's proposition of AI as software is challenged, and the temptation of making many domain-specific AI machines emerges. Thanks to Deep Learning, software can stay decoupled from the computing machinery in the language of linear algebra, which it has in common with supercomputing. A new universal computer for AI understands such language natively to then become a Native Supercomputer. AI has been and will still be the inspiration for computing. The quest to make machines think continues amid the slowdown of Moore's Law. AI might not only maximize the remaining benefits of Moore's Law, but also revive Moore's Law beyond current technology.

The fifth-generation (5G) mobile communication networks, which are anticipated to be soft, green, and super-fast, may possibly be deployed in 2020s to satisfy the challenging demands of mobile communication in various scenarios. Characterized by a mixed set of key performance indicators like data rates, latency, mobility, energy efficiency, and traffic density, 5G services demand a fundamental revolution on the end to end network architecture and key technologies design. Toward a “soft, green, and super-fast” 5G, this paper presents seven innovative 5G R&D themes of China Mobile, including: (1) rethinking Shannon to start a green journey on wireless systems; (2) rethinking Ring and Young for no more “cells”; (3) rethinking signaling and control to make network applications aware and load aware; (4) rethinking antennas to make base stations invisible via SmarTiles; (5) rethinking spectrum and air interface to enable wireless signals to “dress for the occasion”; (6) rethinking fronthaul (FH) to enable Soft RAN via next-generation FH interface; and (7) rethinking the protocol stack for flexible configurations of diversified access points and optimal baseband function split between the base band unit pool and the Remote Radio Systems.

The advent of mobile devices and media cloud services has led to the unprecedented growth of personal photo collections. One of the fundamental problems in managing the increasing number of photos is automatic image tagging. Image tagging is the task of assigning human-friendly tags to an image so that the semantic tags can better reflect the content of the image and therefore can help users better access that image. The quality of image tagging depends on the quality of concept modeling which builds a mapping from concepts to visual images. While significant progresses are made in the past decade on image tagging, the previous approaches can only achieve limited success due to the limited concept representation ability from hand-crafted features (e.g., Scale-Invariant Feature Transform, GIST, Histogram of Oriented Gradients, etc.). Further progresses are made, since the efficient and effective deep learning algorithms have been developed. The purpose of this paper is to categorize and evaluate different image tagging approaches based on deep learning techniques. We also discuss the relevant problems and applications to image tagging, including data collection, evaluation metrics, and existing commercial systems. We conclude the advantages of different image tagging paradigms and propose several promising research directions for future works.

The learning tools necessary to prepare the next generation of students must be shaped by the socio-economic needs of the 21st century. The needs of the 21st century – from rebuilding city scale physical infrastructure to personalized healthcare – not only require learning from the wealth of global information available on the Internet, but also the building of a strong grounding in fundamentals. History has shown that the depth in fundamentals has been achieved through conventional books. Indeed, authoritative books in physical fundamentals have been penned in the 19th century and early 20th century. We present a 21st century cyber-physical learning platform that combines the best of physical books with information systems. The systemic instantiation of the platform combines modern optics and computing to view books, scan objects, and enable interactive learning – while simultaneously benefitting from the vast pool of information on the Internet. The hybrid learning platform preserves the best of the past to pave the way for the future. It also enables future research such as meta-data and descriptive tagging of the large number of images available on the web.

In this paper, we present the argument in favor of an open source, a royalty-free video codec that will keep pace with the evolution of video traffic. Additionally, we argue that the availability of a state-of-the-art, royalty-free codec levels the playing field, allowing small content owners, and application developers to compete with the larger companies that operate in this space.