Skip to main content
×
×
Home

Visual quality assessment: recent developments, coding applications and future trends

  • Tsung-Jung Liu (a1), Yu-Chieh Lin (a1), Weisi Lin (a2) and C.-C. Jay Kuo (a1)
Abstract

Research on visual quality assessment has been active during the last decade. In this work, we provide an in-depth review of recent developments in the field. As compared with existing survey papers, our current work has several unique contributions. First, besides image quality databases and metrics, we put equal emphasis on video quality databases and metrics as this is a less investigated area. Second, we discuss the application of visual quality evaluation to perceptual coding as an example for applications. Third, we benchmark the performance of state-of-the-art visual quality metrics with experiments. Finally, future trends in visual quality assessment are discussed.

    • Send article to Kindle

      To send this article to your Kindle, first ensure no-reply@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about sending to your Kindle. Find out more about sending to your Kindle.

      Note you can select to send to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be sent to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

      Find out more about the Kindle Personal Document Service.

      Visual quality assessment: recent developments, coding applications and future trends
      Available formats
      ×
      Send article to Dropbox

      To send this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Dropbox.

      Visual quality assessment: recent developments, coding applications and future trends
      Available formats
      ×
      Send article to Google Drive

      To send this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Google Drive.

      Visual quality assessment: recent developments, coding applications and future trends
      Available formats
      ×
Copyright
The online version of this article is published within an Open Access environment subject to the conditions of the Creative Commons Attribution-NonCommercial-ShareAlike license . The written permission of Cambridge University Press must be obtained for commercial re-use.
Corresponding author
Corresponding author: C.-C. Jay Kuo Email: cckuo@sipi.usc.edu
References
Hide All
[1]A57 Database. [Online]. Available: http://foulard.ece.cornell.edu/dmc27/vsnr/vsnr.html.
[2]Categorical Image Quality (CSIQ) Database. [Online]. Available: http://vision.okstate.edu/csiq.
[3]Digital Video Library. [Online]. Available: http://www.cdvl.org/.
[4]EPFL-PoliMI Video Quality Assessment Database. [Online]. Available: http://vqa.como.polimi.it/.
[5]IRCCyN/IVC 1080i Database. [Online]. Available: http://www.irccyn.ec-nantes.fr/spip.php?article541.
[6]IRCCyN/IVC SD RoI Database. [Online]. Available: http://www.irccyn.ec-nantes.fr/spip.php?article551.
[7]IVC Image Quality Database. [Online]. Available: http://www2.irccyn.ec-nantes.fr/ivcdb.
[8]IVC-LAR Database. [Online]. Available: http://www.irccyn.ec-nantes.fr/~autrusse/Databases/LAR.
[9]LIVE Image Quality Assessment Database. [Online]. Available: http://live.ece.utexas.edu/research/quality/subjective.htm.
[10]LIVE Video Quality Database. [Online]. Available: http://live.ece.utexas.edu/research/quality/live_video.html.
[11]LIVE Wireless Video Quality Assessment Database. [Online]. Available: http://live.ece.utexas.edu/research/quality/live_wireless_video.html.
[12]MMSP 3D Image Quality Assessment Database. [Online]. Available: http://mmspg.epfl.ch/cms/page-58394.html.
[13]MMSP 3D Video Quality Assessment Database. [Online]. Available: http://mmspg.epfl.ch/3dvqa.
[14]MMSP Scalable Video Database. [Online]. Available: http://mmspg.epfl.ch/svd.
[15]Tampere Image Database. [Online]. Available: http://www.ponomarenko.info/tid2008.htm.
[16]Toyoma Database. [Online]. Available: http://mict.eng.u-toyama.ac.jp/mictdb.html.
[17]VQEG FRTV Phase I Database, 2000. [Online]. Available: ftp://ftp.crc.ca/crc/vqeg/TestSequences/.
[18]VQEG HDTV Database. [Online]. Available: http://www.its.bldrdoc.gov/vqeg/projects/hdtv/.
[19]Wireless Imaging Quality (WIQ) Database. [Online]. Available: http://www.bth.se/tek/rcg.nsf/pages/wiq-db.
[20]Methodology for the Subjective Assessment of the Quality of Television Pictures. Recommendation ITU-R BT.500-11 (2002).
[21]Objective perceptual video quality measurement techniques for digital cable television in the presence of a full reference. Recommendation ITU-T J.144 (Feb. 2004).
[22]Objective perceptual video quality measurement techniques for standard definition digital broadcast television in the presence of a full reference. Recommendation ITU-R BT.1683 (Jan. 2004).
[23]Subjective Video Quality Assessment Methods for Multimedia Applications. Recommendation ITU-T P.910 (Sep. 1999).
[24]Amirshahi, S.A.; Larabi, M.: Spatial–temporal video quality metric based on an estimation of QoE. in Quality of Multimedia Experience (QoMEX), 2011 Third International Workshop on (2011), IEEE, pp. 8489.
[25]Barkowsky, M.; Bialkowski, J.; Eskofier, B.; Bitto, R.; Kaup, A.: Temporal trajectory aware video quality measure. Selected Topics in Signal Processing, IEEE J., 3(2) (2009), 266279.
[26]Bosch, M.; Zhu, F.; Delp, E.J.: Segmentation-based video compression using texture and motion models. Selected Topics in Signal Processing, IEEE J., 5(7) (2011), 13661377.
[27]Chandler, D.M.; Hemami, S.S.: VSNR: A wavelet-based visual signal-to-noise ratio for natural images. Image Processing, IEEE Trans., 16(9) (2007), 22842298.
[28]Channappayya, S.S.; Bovik, A.C.; Heath, R.W.: Rate bounds on ssim index of quantized images. Image Processing, IEEE Trans., 17(9) (2008), 16241639.
[29]Chen, Z.; Guillemot, C.: Perceptually-friendly H.264/avc video coding based on foveated just-noticeable-distortion model. Circuits and Systems for Video Technology, IEEE Trans., 20(6) (2010), 806819.
[30]Choi, M.G.; Jung, J.H.; Jeon, J.W.: No-reference image quality assessment using blur and noise, in proceedings of world academy of science, engineering and technology 50, 2009.
[31]Cui, Z.; Zhu, X.; Subjective quality optimized intra mode selection for H.264i frame coding based on ssim. in Image and Graphics (ICIG), 2011 Sixth International Conference on (2011), IEEE, pp. 157162.
[32]Daly, S.J.: Visible differences predictor: an algorithm for the assessment of image fidelity. In SPIE/IS&T 1992 Symposium on Electronic Imaging: Science and Technology (1992), International Society for Optics and Photonics, pp. 215.
[33]Damera-Venkata, N.; Kite, T.D.; Geisler, W.S.; Evans, B.L.; Bovik, A.C.: Image quality assessment based on a degradation model. Image Processing, IEEE Trans., 9(4) (2000), 636650.
[34]Egiazarian, K.; Astola, J.; Ponomarenko, N.; Lukin, V.; Battisti, F.; Carli, M.: New full-reference quality metrics based on hvs. in CD-ROM Proc. Second Int. Workshop Video Processing and Quality Metrics (2006).
[35]Engelke, U.; Zepernick, H.-J.: Perceptual-based quality metrics for image and video services: A survey. in Next Generation Internet Networks, 3rd EuroNGI Conf. on (2007), IEEE, pp. 190197.
[36]Ferzli, R.; Karam, L.J.: A no-reference objective image sharpness metric based on the notion of just noticeable blur (JNB). Image Processing, IEEE Trans., 18(4) (2009), 717728.
[37]Frater, M.R.; Arnold, J.F.; Vahedian, A.: Impact of audio on subjective assessment of video quality in videoconferencing applications. Circuits and Systems for Video Technology, IEEE Trans., 11(9) (2001), 10591062.
[38]Furini, M.; and Ghini, V.: A video frame dropping mechanism based on audio perception. in Global Telecommunications Conference Workshops, 2004. GlobeCom Workshops 2004. IEEE (2004), IEEE, pp. 211216.
[39]Gao, X.; Lu, W.; Tao, D.; Li, X.: Image quality assessment based on multiscale geometric analysis. Image Processing, IEEE Trans., 18(7) (2009), 14091423.
[40]Ghinea, G.; Thomas, J.P.: Quality of perception: user quality of service in multimedia presentations. Multimedia, IEEE Trans., 7(4) (2005), 786789.
[41]Hontzsch, I.; Karam, L.J.: Adaptive image coding with perceptual distortion control. Image Processing, IEEE Trans., 11(3) (2002), 213222.
[42]Huang, Y.-H.; Ou, T.-S.; Su, P.-Y.; Chen, H.H.: Perceptual rate-distortion optimization using structural similarity index as quality metric. Circuits and Systems for Video Technology, IEEE Trans., 20(11) (2010), 16141624.
[43]Jayant, N.; Johnston, J.; Safranek, R.: Signal compression based on models of human perception. Proceedings of the IEEE, 81(10) (1993), 13851422.
[44]Jin, L.; Egiazarian, K.; Kuo, C.-C.J.: Perceptual image quality assessment using block-based multi-metric fusion (BMMF). in Acoustics, Speech and Signal Processing (ICASSP), 2012 IEEE Int. Conf. (2012), IEEE, pp. 11451148.
[45]Jin, L.; Ponomarenko, N.; Egiazarian, K.: Novel image quality metric based on similarity. in Signals, Circuits and Systems (ISSCS), 2011 10th Int. Symp. (2011), IEEE, pp. 14.
[46]Kawayoke, Y.; Horita, Y.: NR objective continuous video quality assessment model based on frame quality measure. in Image Processing, 2008. ICIP 2008. 15th IEEE Int. Conf. (2008), IEEE, pp. 385388.
[47]Khan, A.; Li, Z.; Sun, L.; Ifeachor, E.: Audiovisual quality assessment for 3G networks in support of e-healthcare services. in Proc. 3rd Int. Conf. Comput. Intell. Med. Healthcare (2007), Citeseer.
[48]Larson, E.C.; Chandler, D.M.; Most apparent distortion: full-reference image quality assessment and the role of strategy. J. Electron. Imaging, 19(1) (2010), 011006–011006.
[49]Lee, C.; Cho, S.; Choe, J.; Jeong, T.; Ahn, W.; Lee, E.: Objective video quality assessment. Opt. Eng., 45(1) (2006), 017004–017004.
[50]Lee, J.-S.; Ebrahimi, T.: Efficient video coding in H.264/avc by using audio-visual information. in Multimedia Signal Processing, 2009. MMSP'09. IEEE Int. Workshop on (2009), IEEE, pp. 16.
[51]Li, C.; Bovik, A.C.: Three-component weighted structural similarity index. in IS&T/SPIE Electronic Imaging (2009), Int. Soc. Opt. Photon., pp. 72420Q72420Q.
[52]Li, Q.; Wang, Z.: Reduced-reference image quality assessment using divisive normalization-based image representation. Selected Topics in Signal Processing, IEEE J., 3(2) (2009), 202211.
[53]Li, S.; Ma, L.; Ngan, K.N.: Video quality assessment by decoupling additive impairments and detail losses. in Quality of Multimedia Experience (QoMEX), 2011 Third Int. Workshop on (2011), IEEE, pp. 9095.
[54]Li, S.; Ma, L.; Zhang, F.; Ngan, K.N.: Temporal inconsistency measure for video quality assessment. in Picture Coding Symposium (PCS), 2010 (2010), IEEE, pp. 590593.
[55]Li, Z.; Qin, S.; Itti, L.: Visual attention guided bit allocation in video compression. Image Vision Comput. 29(1) (2011), 114.
[56]Lin, W.; Kuo, C.-C.J.: Perceptual visual quality metrics: A survey. J. Visual Commun. Image Represent., 22(4) (2011), 297312.
[57]Liu, T.-J.; Lin, W.; Kuo, C.-C.J.: A multi-metric fusion approach to visual quality assessment. in Quality of Multimedia Experience (QoMEX), 2011 Third Int. Workshop on (2011), IEEE, pp. 7277.
[58]Liu, T.-J.; Lin, W.; Kuo, C.-C.J.: A fusion approach to video quality assessment based on temporal decomposition. in Signal & Information Processing Association Annual Summit and Conference (APSIPA ASC), 2012 Asia-Pacific (2012), IEEE, pp. 15.
[59]Liu, T.-J.; Lin, W.; Kuo, C.-C.J.: Image quality assessment using multi-method fusion. Image Processing, IEEE Trans., 22(5) (2013), 17931807.
[60]Liu, T.-J.; Liu, K.-H.; Liu, H.-H.: Temporal information assisted video quality metric for multimedia. in Multimedia and Expo (ICME), 2010 IEEE Int. Conf. (2010), IEEE, pp. 697701.
[61]Lu, Z.; Lin, W.; Yang, X.; Ong, E.; Yao, S.: Modeling visual attention's modulatory aftereffects on visual sensitivity and quality evaluation. Image Processing, IEEE Trans., 14(11) (2005), 19281942.
[62]Lubin, J.: A visual discrimination model for imaging system design and evaluation. Vision Models Target Detect. Recogn., 2 ( 1995), 245357.
[63]Luo, H.: A training-based no-reference image quality assessment algorithm. in Image Processing, 2004. ICIP'04. 2004 Int. Conf. (2004), vol. 5, IEEE, pp. 29732976.
[64]Marziliano, P.; Dufaux, F.; Winkler, S.; Ebrahimi, T.: A no-reference perceptual blur metric. in Image Processing. 2002. Proceedings. 2002 Int. Conf. (2002), vol. 3, IEEE, pp. III–57.
[65]Masry, M.; Hemami, S.S.; Sermadevi, Y.: A scalable wavelet-based video distortion metric and applications. Circuits and Systems for Video Technology, IEEE Trans., 16(2) (2006), 260273.
[66]Masry, M.A.; Hemami, S.S.: A metric for continuous quality evaluation of compressed video with severe distortions. Signal process.: Image commun., 19(2) (2004), 133146.
[67]Naccari, M.; Pereira, F.: Advanced H.264/avc-based perceptual video coding: Architecture, tools, and assessment. Circuits and Systems for Video Technology, IEEE Trans., 21(6) (2011), 766782.
[68]Narwaria, M.; Lin, W.: Objective image quality assessment based on support vector regression. Neural Networks, IEEE Trans., 21(3) (2010), 515519.
[69]Narwaria, M.; Lin, W.: Machine learning based modeling of spatial and temporal factors for video quality assessment. in Image Processing (ICIP), 2011 18th IEEE Int. Conf. (2011), IEEE, pp. 25132516.
[70]Narwaria, M.; Lin, W.: Video quality assessment using temporal quality variations and machine learning. in Multimedia and Expo (ICME), 2011 IEEE Int. Conf. (2011), IEEE, pp. 16.
[71]Ndjiki-Nya, P.; Stuber, C.; Wiegand, T.: Texture synthesis method for generic video sequences. in Image Processing, 2007. ICIP 2007. IEEE Int. Conf. (2007), vol. 3, IEEE, pp. III–397.
[72]Ninassi, A.; Le Meur, O.; Le Callet, P.; Barba, D.: Considering temporal variations of spatial visual distortions in video quality assessment. Selected Topics in Signal Processing, IEEE J., 3(2) (2009), 253265.
[73]Oh, B.T.; Su, Y.; Segall, C.; Kuo, C.-C.: Synthesis-based texture video coding with side information. Circuits and Systems for Video Technology, IEEE Trans., 21(5) (2011), 647659.
[74]Ong, E.; Lin, W.; Lu, Z.; Yao, S.; Yang, X.; Jiang, L.: No-reference JPEG-2000 image quality metric. in Multimedia and Expo, 2003. ICME'03. Proc. 2003 Int. Conf. (2003), vol. 1, IEEE, pp. I–545.
[75]Ou, T.-S.; Huang, Y.-H.; Chen, H.H.: SSIM-based perceptual rate control for video coding. Circuits and Systems for Video Technology, IEEE Trans., 21(5) (2011), 682691.
[76]Park, J.; Seshadrinathan, K.; Lee, S.; Bovik, A.C.: Video quality pooling adaptive to perceptual distortion severity, IEEE Trans. on image processsing, 22(2) (2013), 610620.
[77]Peli, E.: Contrast in complex images. JOSA A, 7(10) (1990), 20322040.
[78]Pinson, M.H.; Wolf, S.: A new standardized method for objectively measuring video quality. Broadcasting, IEEE Trans., 50(3) (2004), 312322.
[79]Ponomarenko, N.; Silvestri, F.; Egiazarian, K.; Carli, M.; Astola, J.; Lukin, V.: On between-coefficient contrast masking of dct basis functions. in Proc. Third Int. Workshop on Video Processing and Quality Metrics (2007), vol. 4.
[80]Rehman, A.; Wang, Z.: Reduced-reference ssim estimation. In Image Processing (ICIP), 2010 17th IEEE Int. Conf. (2010), IEEE, pp. 289292.
[81]Richter, T.; Kim, K.J.: A ms-ssim optimal jpeg 2000 encoder. in Data Compression Conference, 2009. DCC'09. (2009), IEEE, pp. 401410.
[82]Seshadrinathan, K.; Bovik, A.C.: Motion tuned spatio-temporal quality assessment of natural videos. Image Processing, IEEE Trans., 19(2) (2010), 335350.
[83]Seshadrinathan, K.; Soundararajan, R.; Bovik, A.C.; and Cormack, L.K.: Study of subjective and objective quality assessment of video. Image Processing, IEEE Trans., 19(6) (2010), 14271441.
[84]Sheikh, H.R.; Bovik, A.C.: Image information and visual quality. Image Processing, IEEE Trans., 15(2) (2006), 430444.
[85]Sheikh, H.R.; Bovik, A.C.; De Veciana, G.: An information fidelity criterion for image quality assessment using natural scene statistics. Image Processing, IEEE Trans., 14(12) (2005), 21172128.
[86]Sheikh, H.R.; Sabir, M.F.; Bovik, A.C.: A statistical evaluation of recent full reference image quality assessment algorithms. Image Processing, IEEE Trans., 15(11) (2006), 34403451.
[87]Suresh, S.; Babu, V.; Sundararajan, N.: Image quality measurement using sparse extreme learning machine classifier. in Control, Automation, Robotics and Vision, 2006. ICARCV'06. 9th Int. Conf. (2006), IEEE, pp. 16.
[88]Tan, D.; Tan, C.; Wu, H.: Perceptual color image coding with JPEG2000. Image Processing, IEEE Trans., 19(2) (2010), 374– 383.
[89]Teo, P.C.; Heeger, D.J.: Perceptual image distortion. in Image Processing, 1994. Proceedings. ICIP-94., IEEE Int. Conf. (1994), vol. 2, IEEE, pp. 982986.
[90]Tong, H.; Li, M.; Zhang, H.-J.; Zhang, C.: No-reference quality assessment for JPEG2000 compressed images. in Image Processing, 2004. ICIP'04. 2004 Int. Conf. (2004), vol. 5, IEEE, pp. 35393542.
[91]VQEG. Final report from the video quality experts group on the validation of objective models of video quality assessment, phase I. Mar. 2000. [Online]. Available: http://www.its.bldrdoc.gov/vqeg/projects/frtv_phaseI.
[92]VQEG. Final report from the video quality experts group on the validation of objective models of video quality assessment, phase II. Aug. 2003. [Online]. Available: http://www.its.bldrdoc.gov/vqeg/projects/frtv_phaseII.
[93]Vu, P.V.; Vu, C.T.; Chandler, D.M.: A spatiotemporal most-apparent-distortion model for video quality assessment. In Image Processing (ICIP), 2011 18th IEEE Int. Conf. (2011), IEEE, pp. 25052508.
[94]Wang, S.; Rehman, A.; Wang, Z.; Ma, S.; Gao, W.: SSIM-motivated rate-distortion optimization for video coding. Circuits and Systems for Video Technology, IEEE Trans., 22(4) (2012), 516529.
[95]Wang, Z.; Bovik, A.C.: Mean squared error: love it or leave it? a new look at signal fidelity measures. Signal Process. Mag., IEEE, 26(1) (2009), 98117.
[96]Wang, Z.; Bovik, A.C.; Sheikh, H.R.; Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. Image Process. IEEE Trans., 13(4) (2004), 600612.
[97]Wang, Z.; Li, Q.: Video quality assessment using a statistical model of human visual speed perception. JOSA A, 24(12) (2007), B61B69.
[98]Wang, Z.; Li, Q.: Information content weighting for perceptual image quality assessment. Image Process. IEEE Trans., 20(5) (2011), 11851198.
[99]Wang, Z.; Lu, L.; Bovik, A.C.: Video quality assessment based on structural distortion measurement. Signal Process.: Image Commun., 19(2) (2004), 121132.
[100]Wang, Z.; Simoncelli, E.P.: Translation insensitive image similarity in complex wavelet domain. in In Acoustics, Speech, and Signal Processing, 2005. Proceedings (ICASSP05). IEEE Int. Conf. (2005), Citeseer.
[101]Wang, Z.; Simoncelli, E.P.; Bovik, A.C.: Multiscale structural similarity for image quality assessment. in Signals, Systems and Computers, 2003. Conference Record of the Thirty-Seventh Asilomar Conf. (2003), vol. 2, IEEE, pp. 13981402.
[102]Watson, A.B.; Hu, J.; McGowan, J.F.: Digital video quality metric based on human vision. J. Elect. imaging, 10(1) (2001), 2029.
[103]Wei, Z.; Ngan, K.N.: Spatio-temporal just noticeable distortion profile for grey scale image/video in DCT domain. Circuits and Systems for Video Technology, IEEE Trans., 19(3) (2009), 337346.
[104]Winkler, S.: Digital video quality: vision models and metrics. Wiley, 2005.
[105]Winkler, S.; Mohandas, P.: The evolution of video quality measurement: from psnr to hybrid metrics. Broadcasting, IEEE Trans., 54(3) (2008), 660668.
[106]Yang, C.-L.; Leung, R.-K.; Po, L.-M.; Mai, Z.-Y.: An SSIM-optimal H.264/avc inter frame encoder. in Intelligent Computing and Intelligent Systems, 2009. ICIS 2009. IEEE Int. Conf. (2009), vol. 4, IEEE, pp. 291295.
[107]Yim, C.; Bovik, A.C.: Quality assessment of deblocked images. Image Processing, IEEE Trans., 20(1) (2011), 8898.
[108]Zhang, L.; Zhang, L.; Mou, X.; Zhang, D.: FSIM: a feature similarity index for image quality assessment. Image Processing, IEEE Trans., 20(8) (2011), 23782386.
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

APSIPA Transactions on Signal and Information Processing
  • ISSN: 2048-7703
  • EISSN: 2048-7703
  • URL: /core/journals/apsipa-transactions-on-signal-and-information-processing
Please enter your name
Please enter a valid email address
Who would you like to send this to? *
×

Keywords

Metrics

Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed