With the recent development of new three-dimensional (3D) multimedia services such as 3D television or free viewpoint television, a new 3D video format, called multiview video + depth (MVD) is currently being investigated. MVD allows synthesizing as many views as required at the receiver side, thus providing smooth scene transitions and the ability to experience a new 3D perspective with each different viewing point. The format introduces, alongside traditional 2D image sequences, sequences of depth maps, which must be efficiently coded to achieve good quality for the synthesized views. One approach to code depth videos is to exploit the correlations between texture and depth. In this work, we propose a new tool to code depth videos in which the texture Intra modes are inherited and used as predictors for the depth Intra modes, hence reducing the mode signaling bitrate. The tool is only used in prediction units where texture and depth Intra directions, or modes, are expected to match. Two criteria that exploit the statistical dependency between the texture and depth Intra modes are studied in this work: GradientMax and DominantAngle. Average bitrate reductions of 1.3 and 1.6% on synthesized sequences are reported for GradientMax and DominantAngle, respectively. The latter method additionally achieves 2.3% bitrate reduction on depth sequences.

In this invited paper, my overview material on the same topic as presented in the plenary overview session of APSIPA-2011 and the tutorial material presented in the same conference [1] are expanded and updated to include more recent developments in deep learning. The previous and the updated materials cover both theory and applications, and analyze its future directions. The goal of this tutorial survey is to introduce the emerging area of deep learning or hierarchical learning to the APSIPA community. Deep learning refers to a class of machine learning techniques, developed largely since 2006, where many stages of non-linear information processing in hierarchical architectures are exploited for pattern classification and for feature learning. In the more recent literature, it is also connected to representation learning, which involves a hierarchy of features or concepts where higher-level concepts are defined from lower-level ones and where the same lower-level concepts help to define higher-level ones. In this tutorial survey, a brief history of deep learning research is discussed first. Then, a classificatory scheme is developed to analyze and summarize major work reported in the recent deep learning literature. Using this scheme, I provide a taxonomy-oriented survey on the existing deep architectures and algorithms in the literature, and categorize them into three classes: generative, discriminative, and hybrid. Three representative deep architectures – deep autoencoders, deep stacking networks with their generalization to the temporal domain (recurrent networks), and deep neural networks (pretrained with deep belief networks) – one in each of the three classes, are presented in more detail. Next, selected applications of deep learning are reviewed in broad areas of signal and information processing including audio/speech, image/vision, multimodality, language modeling, natural language processing, and information retrieval. Finally, future directions of deep learning are discussed and analyzed.

In this paper, preamble design for estimation and compensation of channel distortion parameters (or channel impairments) in orthogonal frequency-division multiplexing (OFDM) transmission over peak-limited channels is studied. Specifically, the designed preamble considers the estimation of frequency selective channels, carrier frequency offset (CFO), in-phase/quadrature-phase (I/Q) imbalance together with the minimization of peak-to-average power ratio (PAPR) of the transmitted signals. In the proposed design, we employ adaptive Markov chain Monte Carlo (AMCMC) techniques to select preamble sequence that minimizes the channel estimate mean-squared error while suppressing the effect of the I/Q mismatch. AMCMC algorithm is also deployed to select phase information to the designed preamble in order to minimize the PAPR of the oversampled preamble signals in time domain. To estimate CFO, maximum likelihood-based scheme that utilizes two successive OFDM preambles is employed, and the CFO is estimated by considering phase rotation between two consecutive received OFDM preambles. Numerical simulations are provided to verify the efficacy of the proposed design.

This paper considers the placement of m sensors at n > m possible locations. Given noisy observations, knowledge of the state correlation matrix, and a mean-square error criterion (equivalently maximizing an efficacy cost criterion), the problem is formulated as an integer programming problem. Computing the solution for large m and n is infeasible, requiring us to look at approximate algorithms and bounding optimal performance. Approximate algorithms include greedy algorithms and variations based on examining the efficacy cost function and projection-based methods that all run in polynomial time of m and n. A sequence of nested bounds are found that upper bound the optimal performance (with analysis based on using matrix pencils and generalized eigenvectors). Finally, we show through simulations that the approximate algorithms perform well and provide tight implementable lower bounds to optimal performance and the nested bounds provide upper bounds to optimal performance with tighter bounds achieved with increasing complexity. The sensor placement problem has many energy applications where we are often confronted with limited resources. Some examples include where to place environmental sensors for an area in which there are many distributed solar photovoltaic generators and where to place grid monitors on an electrical distribution microgrid.

Motivated by the inherent correlation between the speech features and their lexical words, we propose in this paper a new framework for learning the parameters of the corresponding acoustic and language models jointly. The proposed framework is based on discriminative training of the models' parameters using minimum classification error criterion. To verify the effectiveness of the proposed framework, a set of four large decoding graphs is constructed using weighted finite-state transducers as a composition of two sets of context-dependent acoustic models and two sets of n-gram-based language models. The experimental results conducted on this set of decoding graphs validated the effectiveness of the proposed framework when compared with four baseline systems based on maximum likelihood estimation and separate discriminative training of acoustic and language models in benchmark testing of two speech corpora, namely TIMIT and RM1.

Cloud Computing has become a well-known primitive nowadays; many researchers and companies are embracing this fascinating technology with feverish haste. In the meantime, security and privacy challenges are brought forward while the number of cloud storage user increases expeditiously. In this work, we conduct an in-depth survey on recent research activities of cloud storage security in association with cloud computing. After an overview of the cloud storage system and its security problem, we focus on the key security requirement triad, i.e., data integrity, data confidentiality, and availability. For each of the three security objectives, we discuss the new unique challenges faced by the cloud storage services, summarize key issues discussed in the current literature, examine, and compare the existing and emerging approaches proposed to meet those new challenges, and point out possible extensions and futuristic research opportunities. The goal of our paper is to provide a state-of-the-art knowledge to new researchers who would like to join this exciting new field.

The bandwidth limitation of wideband (WB) audio systems degrades the subjective quality and naturalness of audio signals. In this paper, a new method for blind bandwidth extension of WB audio signals is proposed based on non-linear prediction and hidden Markov model (HMM). The high-frequency (HF) components in the band of 7–14 kHz are artificially restored only from the low-frequency information of the WB audio. State-space reconstruction is used to convert the fine spectrum of WB audio to a multi-dimensional space, and a non-linear prediction based on nearest-neighbor mapping is employed in the state space to restore the fine spectrum of the HF components. The spectral envelope of the resulting HF components is estimated based on an HMM according to the features extracted from the WB audio. In addition, the proposed method and the reference methods are applied to the ITU-T G.722.1 WB audio codec for comparison with the ITU-T G.722.1C super WB audio codec. Objective quality evaluation results indicate that the proposed method is preferred over the reference bandwidth extension methods. Subjective listening results show that the proposed method has a comparable audio quality with G.722.1C and improves the extension performance compared with the reference methods.

While demand response has achieved promising results on making the power grid more efficient and reliable, the additional dynamics and flexibility brought by demand response also increase the uncertainty and complexity of the centralized load forecast. In this paper, we propose a game-theoretic demand response scheme that can transform the traditional centralized load prediction structure into a distributed load prediction system by the participation of customers. Moreover, since customers are generally rational and thus naturally selfish, they may cheat if cheating can improve their payoff. Therefore, enforcing truth-telling is crucial. We prove analytically and demonstrate with simulations that the proposed game-theoretic scheme is incentive compatible, i.e., all customers are motivated to report and consume their true optimal demands and any deviation will lead to a utility loss. We also prove theoretically that the proposed demand response scheme can lead to the solution that maximizes social welfare and is proportionally fair in terms of utility function. Moreover, we propose a simple dynamic pricing algorithm for the power substation to control the total demand of all customers to meet the target demand curve. Finally, simulations are shown to demonstrate the efficiency and effectiveness of the proposed game-theoretic algorithm.

A tone mapping operation (TMO) for HDR images with fixed-point arithmetic is proposed. A TMO generates a low dynamic range (LDR) image from a high dynamic range (HDR) image by compressing its dynamic range. Since HDR images are generally expressed in a floating-point data format, a TMO also deals with floating-point data even though resulting LDR images have integer data. As a result, conventional TMOs require many resources such as computational and memory cost. To reduce the resources, an integer TMO which treats a floating-point number as two 8-bit integer numbers was proposed. However, this method has the limitation of available input HDR image formats. The proposed method introduces an intermediate format to relieve the limitation of input formats, and expands the integer TMO for the intermediate format. The proposed integer TMO can be applied for multiple formats such as the RGBE and the OpenEXR. Moreover, the method can conduct all calculations in the TMO with fixed-point arithmetic. Using both integer data and fixed-point arithmetic, the method reduces not only the memory cost, but also the computational cost. The experimental and evaluation results show that the proposed method reduces the computational and memory cost, and gives almost same quality of LDR images, compared with the conventional method with floating-point arithmetic.

With the introduction of enhanced metering and communication capabilities in smart grids, utility companies will have the ability to extend Demand Response (DR) to small customers through Time-Dependent Pricing (TDP). By using pricing signals that more accurately reflect the demand-supply situation of an electricity network, utility companies can induce customers to shift their consumptions to off-peak periods, thus reducing the cost and improving the reliability of the network. Despite its promises, large scale deployment of DR still faces many obstacles, in particular, resistance from customers due to concerns over cost, uncertain price and privacy issues. In this paper, we propose a dual-price DR scheme to overcome some of these issues. The proposed scheme offers both regulated flat price and TDP to customers to meet their different risk-taking profiles. The TDP rates are computed from a cost minimization problem considering both consumption behaviours of customers and generation cost. We also present an analysis for solving the optimization problem and find a closed form solution for TDP. It is shown that the proposed scheme is effective in inducing the desired consumption behaviours. In addition, it is found that with proper price signals, the proposed scheme can provide incentives to both utility companies and TDP customers, thus encouraging the adoption of TDP. Theoretical results from this paper are illustrated using numerical examples.

Emotion recognition is the ability to identify what people would think someone is feeling from moment to moment and understand the connection between his/her feelings and expressions. In today's world, human–computer interaction (HCI) interface undoubtedly plays an important role in our daily life. Toward harmonious HCI interface, automated analysis and recognition of human emotion has attracted increasing attention from the researchers in multidisciplinary research fields. In this paper, a survey on the theoretical and practical work offering new and broad views of the latest research in emotion recognition from bimodal information including facial and vocal expressions is provided. First, the currently available audiovisual emotion databases are described. Facial and vocal features and audiovisual bimodal data fusion methods for emotion recognition are then surveyed and discussed. Specifically, this survey also covers the recent emotion challenges in several conferences. Conclusions outline and address some of the existing emotion recognition issues.

The new High Efficiency Video Coding (HEVC) standard achieves higher encoding efficiency when compared to its predecessors such as H.264/AVC. One of the factors responsible for this improvement is the new intra prediction method, which introduces a larger number of prediction directions resulting in an enhanced rate-distortion (RD) performance obtained at the cost of higher computational complexity. This paper proposes an algorithm to accelerate the intra mode decision, reducing the complexity of intra coding. The acceleration procedure takes into account the texture local directionality information and explores the correlation of intra modes across levels of the hierarchical tree structure used in HEVC. Experimental results show that the proposed algorithm provides a decrease of 39.22 and 43.88% in the HEVC intra prediction processing time on average, for all-intra high efficiency (AI-HE) and low complexity (AI-LC) configurations, respectively, with a small degradation in encoding efficiency (BD-PSNR loss of 0.1 dB for AI-HE and 0.8 dB for AI-LC on average).

Although research in audio recognition has traditionally focused on speech and music signals, the problem of environmental sound recognition (ESR) has received more attention in recent years. Research on ESR has significantly increased in the past decade. Recent work has focused on the appraisal of non-stationary aspects of environmental sounds, and several new features predicated on non-stationary characteristics have been proposed. These features strive to maximize their information content pertaining to signal's temporal and spectral characteristics. Furthermore, sequential learning methods have been used to capture the long-term variation of environmental sounds. In this survey, we will offer a qualitative and elucidatory survey on recent developments. It includes four parts: (i) basic environmental sound-processing schemes, (ii) stationary ESR techniques, (iii) non-stationary ESR techniques, and (iv) performance comparison of selected methods. Finally, concluding remarks and future research and development trends in the ESR field will be given.

Multi-view images are acquired by a lensless compressive imaging architecture, which consists of an aperture assembly and multiple sensors. The aperture assembly consists of a two-dimensional array of aperture elements whose transmittance can be individually controlled to implement a compressive sensing matrix. For each transmittance pattern of the aperture assembly, each of the sensors takes a measurement. The measurement vectors from the multiple sensors represent multi-view images of the same scene. We present theoretical framework for multi-view reconstruction and experimental results for enhancing quality of image using compressive measurements from multiple sensors.

This paper presents a novel rate control scheme designed for the newest high efficiency video coding (HEVC) standard, and aimed at enhancing the quality of regions of interest (ROI) for a videoconferencing system. It is designed to consider the different regions at both frame level and coding tree unit (CTU) level. The proposed approach allocates a higher bit rate to the region of interest while keeping the global bit rate close to the assigned target value. The ROIs, typically faces in this application, are automatically detected and each CTU is classified in a region of interest map. This binary map is given as input to the rate control algorithm and the bit allocation is made accordingly. The algorithm is tested, first, using the initial version of the controller introduced in HEVC test model (HM.10), then, extended in HM.13. In this work, we first investigate the impact of differentiated bit allocation between the two regions using a fixed bit rate ratio in intra-coded frames (I-frames) and Bidirectionally predicted frames (B-frames). Then, unit quantization parameters (QPs) are computed independently for CTUs of different regions. The proposed approach has been compared to the reference controller implemented in HM and to a ROI-based rate control algorithm initially proposed for H.264 that we adopted to HEVC and implemented in HM.9. Experimental results show that our scheme has comparable performances with the ROI-based controller proposed for H.264. It achieves accurate target bit rates and provides an improvement in region of interest quality, both in objective metrics (up to 2 dB in PSNR) and based on subjective quality evaluation.

A speaker verification system automatically accepts or rejects a claimed identity of a speaker based on a speech sample. Recently, a major progress was made in speaker verification which leads to mass market adoption, such as in smartphone and in online commerce for user authentication. A major concern when deploying speaker verification technology is whether a system is robust against spoofing attacks. Speaker verification studies provided us a good insight into speaker characterization, which has contributed to the progress of voice conversion technology. Unfortunately, voice conversion has become one of the most easily accessible techniques to carry out spoofing attacks; therefore, presents a threat to speaker verification systems. In this paper, we will briefly introduce the fundamentals of voice conversion and speaker verification technologies. We then give an overview of recent spoofing attack studies under different conditions with a focus on voice conversion spoofing attack. We will also discuss anti-spoofing attack measures for speaker verification.

Systems with hundreds of microphones for acoustic field acquisition, or hundreds of loudspeakers for rendering, have been proposed and built. To analyze, design, and apply such systems requires a framework that allows us to leverage the vast set of tools available in digital signal processing in order to achieve intuitive and efficient algorithms. We thus propose a discrete space–time framework, grounded in classical acoustics, which addresses the discrete nature of the spatial and temporal sampling. In particular, a short-space/time Fourier transform is introduced, which is the natural extension of the localized or short-time Fourier transform. Processing in this intuitive domain allows us to easily devise algorithms for beam-forming, source separation, and multi-channel compression, among other useful tasks. The essential space band-limitedness of the Fourier spectrum is also used to solve the spatial equalization task required for sound field rendering in a region of interest. Examples of applications are shown.

The analysis of JPEG double-compressed images is a problem largely studied by the multimedia forensics community, as it might be exploited, e.g., for tampering localization or source device identification. In many practical scenarios, like photos uploaded on blogs, on-line albums, and photo sharing web sites, images might be JPEG compressed several times. However, the identification of the number of compression stages applied to an image remains an open issue. We proposes a forensic method based on the analysis of the distribution of the first significant digits of the discrete cosine transform coefficients, which follow Benford's law in images compressed just once. Then, the detector is optimized and extended in order to identify accurately the number of compression stages applied to an image. The experimental validation considers up to four consecutive compression stages and shows that the proposed approach extends and outperforms the previously-published algorithms for double JPEG compression detection.

A sound reproduction system usually consists of several types of loudspeakers to cater to sophisticated applications. The directivity of a loudspeaker is a measure of its efficiency in sending sounds to a particular direction instead of all directions. Demand to control the directivity of a sound reproduction system is gaining momentum with many new designs of directional loudspeakers, including the acoustic dome, horn loudspeaker, loudspeaker array, and parametric array loudspeaker (PAL). The PAL is an application of the parametric acoustic array in air, which generates a sound beam from the interaction of ultrasonic beams. The PAL has several desired features, such as its narrow beamwidth over a wide frequency range, low sound attenuation over a long distance, and ability to reproduce perceptually near sound images. The PAL is also advantageous in using a smaller driving unit to transmit an equally narrow sound beam as compared to the conventional loudspeaker and broadside loudspeaker array. An overview of directivity control methods of the PAL is presented in this paper. In particular, acoustic models and signal processing techniques in controlling the directivity of the PAL are emphasized.