A survey of content-based image retrieval with high-level semantics

Aus de_evolutionary_art_org
Wechseln zu: Navigation, Suche


Liu, D. Zhang, G. Lu and W.-Y. Ma: A survey of content-based image retrieval with high-level semantics. Pattern Recogn., vol. 40, no. 1, pp. 262-282, 2007




In order to improve the retrieval accuracy of content-based image retrieval systems, research focus has been shifted from designing sophisticated low-level feature extraction algorithms to reducing the ‘semantic gap’ between the visual features and the richness of human semantics. This paper attempts to provide a comprehensive survey of the recent technical achievements in high-level semantic-based image retrieval. Major recent publications are included in this survey covering different aspects of the research in this area, including low-level image feature extraction, similarity measurement, and deriving high-level semantic features. We identify five major categories of the state-of-the-art techniques in narrowing down the ‘semantic gap’: (1) using object ontology to define high-level concepts; (2) using machine learning methods to associate low-level features with query concepts; (3) using relevance feedback to learn users’ intention; (4) generating semantic template to support high-level image retrieval; (5) fusing the evidences from HTML text and the visual content of images for WWW image retrieval. In addition, some other related issues such as image test bed and retrieval performance evaluation are also discussed. Finally, based on existing technology and the demand from real-world applications, a few promising future research directions are suggested.

Extended Abstract


title = "A survey of content-based image retrieval with high-level semantics ",
journal = "Pattern Recognition ",
volume = "40",
number = "1",
pages = "262 - 282",
year = "2007",
note = "",
issn = "0031-3203",
doi = "http://dx.doi.org/10.1016/j.patcog.2006.04.045",
url = "http://www.sciencedirect.com/science/article/pii/S0031320306002184 http://de.evo-art.org/index.php?title=A_survey_of_content-based_image_retrieval_with_high-level_semantics",
author = "Ying Liu and Dengsheng Zhang and Guojun Lu and Wei-Ying Ma",
keywords = "Content-based image retrieval",
keywords = "Semantic gap",
keywords = "High-level semantics",
keywords = "Survey "

Used References

[1] J. Eakins, M. Graham, Content-based image retrieval, Technical Report, University of Northumbria at Newcastle, 1999.

[2] I.K. Sethi, I.L. Coman, Mining association rules between low-level image features and high-level concepts, Proceedings of the SPIE Data Mining and Knowledge Discovery, vol. III, 2001, pp. 279–290.

[3] S.K. Chang, S.H. Liu, Picture indexing and abstraction techniques for pictorial databases, IEEE Trans. Pattern Anal. Mach. Intell. 6 (4) (1984) 475–483.

[4] C. Faloutsos, R. Barber, M. Flickner, J. Hafner, W. Niblack, D. Petkovic, W. Equitz, Efficient and effective querying by image content, J. Intell. Inf. Syst. 3 (3–4) (1994) 231–262.

[5] A. Pentland, R.W. Picard, S. Scaroff, Photobook: content-based manipulation for image databases, Int. J. Comput. Vision 18 (3) (1996) 233–254.

[6] A. Gupta, R. Jain, Visual information retrieval, Commun. ACM 40 (5) (1997) 70–79.

[7] J.R. Smith, S.F. Chang, VisualSeek: a fully automatic contentbased query system, Proceedings of the Fourth ACM International Conference on Multimedia, 1996, pp. 87–98.

[8] W.Y. Ma, B. Manjunath, Netra: a toolbox for navigating large image databases, Proceedings of the IEEE International Conference on Image Processing, 1997, pp. 568–571.

[9] J.Z. Wang, J. Li, G. Wiederhold, SIMPLIcity: semantics-sensitive integrated matching for picture libraries, IEEE Trans. Pattern Anal. Mach. Intell. 23 (9) (2001) 947–963.

[10] F. Long, H.J. Zhang, D.D. Feng, Fundamentals of content-based image retrieval, in: D. Feng (Ed.), Multimedia Information Retrieval and Management, Springer, Berlin, 2003.

[11] Y. Rui, T.S. Huang, S.-F. Chang, Image retrieval: current techniques, promising directions, and open issues, J. Visual Commun. Image Representation 10 (4) (1999) 39–62.

[12] A. Mojsilovic, B. Rogowitz, Capturing image semantics with low-level descriptors, Proceedings of the ICIP, September 2001, pp. 18–21.

[13] X.S. Zhou, T.S. Huang, CBIR: from low-level features to highlevel semantics, Proceedings of the SPIE, Image and Video Communication and Processing, San Jose, CA, vol. 3974, January 2000, pp. 426–431.

[14] Y. Chen, J.Z.Wang, R.Krovetz, An unsupervised learning approach to content-based image retrieval, IEEE Proceedings of the International Symposium on Signal Processing and its Applications, July 2003, pp. 197–200.

[15] A.W.M. Smeulders, M. Worring, A. Gupta, R. Jain, Content-based image retrieval at the end of the early years, IEEE Trans. Pattern Anal. Mach. Intell. 22 (12) (2000) 1349–1380.

[16] F. Jing, M. Li, L. Zhang, H.-J. Zhang, B. Zhang, Learning in regionbased image retrieval, Proceedings of the International Conference on Image and Video Retrieval (CIVR2003), 2003, pp. 206–215.

[17] H. Feng, D.A. Castanon, W.C. Karl, A curve evolution approach for image segmentation using adaptive flows, Proceedings of the International Conference on Computer Vision (ICCV’01), 2001, pp. 494–499.

[18] W.Y. Ma, B.S. Majunath, Edge flow: a framework of boundary detection and image segmentation, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 1997, pp. 744–749.

[19] J. Shi, J. Malik, Normalized cuts and image segmentation, IEEE Trans. Pattern Anal. Mach. Intell. (PAMI) 22 (8) (2000) 888–905.

[20] D. Comaniciu, P. Meer, Robust analysis of feature spaces: color image segmentation, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1997, pp. 750–755.

[21] P.L. Stanchev, D. Green Jr., B. Dimitrov, High level color similarity retrieval, Int. J. Inf. Theories Appl. 10 (3) (2003) 363–369.

[22] K.A. Hua, K. Vu, J.-H. Oh, SamMatch: a flexible and efficient sampling-based image retrieval technique for large image databases, Proceedings of the Seventh ACM International Multimedia Conference (ACM Multimedia’99), November 1999, pp. 225–234.

[23] Y. Deng, B.S. Manjunath, Unsupervised segmentation of color-texture regions in images and video, IEEE Trans. Pattern Anal. Mach. Learn. (PAMI) 23 (8) (2001) 800–810.

[24] H. Feng, T.-S. Chua, A boostrapping approach to annotating large image collection, Workshop on Multimedia Information Retrieval in ACM Multimedia, November 2003, pp. 55–62.

[25] Y. Liu, D.S. Zhang, G. Lu, W.-Y. Ma, Region-based image retrieval with perceptual colors, Proceedings of the Pacific-Rim Multimedia Conference (PCM), December 2004, pp. 931–938.

[26] C. Carson, S. Belongie, H. Greenspan, J. Malik, Blobworld: image segmentation using expectation-maximization and its application to image querying, IEEE Trans. Pattern Anal. Mach. Intell. 8 (8) (2002) 1026–1038.

[27] R. Shi, H. Feng, T.-S. Chua, C.-H. Lee, An adaptive image content representation and segmentation approach to automatic image annotation, International Conference on Image and Video Retrieval (CIVR), 2004, pp. 545–554.

[28] C.P. Town, D. Sinclair, Content-based image retrieval using semantic visual categories, Society for Manufacturing Engineers, Technical Report MV01-211, 2001.

[29] J.R. Smith, C.-S. Li, Decoding image semantics using composite region templates, IEEEWorkshop on Content-Based Access of Image and Video Libraries (CBAIVL-98), June 1998, pp. 9–13.

[30] W.K. Leow, S.Y. Lai, Scale and orientation-invariant texture matching for image retrieval, in: M.K. Pietikainen (Ed.), Texture Analysis in Machine Vision, World Scientific, Singapore, 2000.

[31] V. Mezaris, I. Kompatsiaris, M.G. Strintzis, An ontology approach to object-based image retrieval, Proceedings of the ICIP, vol. II, 2003, pp. 511–514.

[32] K.N. Plataniotis, A.N. Venetsanopoulos, Color Image Processing and Applications, Springer, Berlin, 2000.

[33] B.S. Manjunath, et al., Color and texture descriptors, IEEE Trans. CSVT 11 (6) (2001) 703–715.

[34] J.Z. Wang, J. Li, D. Chan, G. Wiederhold, Semantics-sensitive retrieval for digital picture libraries, Digital Library Magazine, vol. 5(11), 1999.

[35] E. Chang, S. Tong, SVMactive-support vector machine active learning for image retrieval, Proceedings of the ACM International Multimedia Conference, October 2001, pp. 107–118.

[36] X. Zheng, D. Cai, X. He, W.-Y. Ma, X. Lin, Locality preserving clustering for image database, Proceedings of the 12th ACM Multimedia, October 2004.

[37] B.S. Manjunath, et al., Introduction to MPEG-7, Wiley, New York, 2002.

[38] T. Gevers, A. Smeulders, Content-based image retrieval by viewpointinvariant color indexing, Image Vision Comput. 17 (1999) 475–488.

[39] W. Wang, Y. Song, A. Zhang, Semantics retrieval by content and context of image regions, Proceedings of the 15th International Conference on Vision Interface (VI’2002), May 2002, pp. 17–24.

[40] K.N. Plataniotis, et al., Adaptive fuzzy systems for multichannel signal processing, Proc. IEEE 87 (9) (1999) 1601–1622.

[41] R. Lukac, et al., Vector filtering for color imaging, IEEE Signal Process. Mag. (2005) 74–86.

[42] P. Stanchev, Using image mining for image retrieval, IASTED Conference “Computer Science and Technology,” Cancun, Mexico, May 2003, pp. 214–218.

[43] H. Tamura, S. Mori, T. Yamawaki, Texture features corresponding to visual perception, IEEE Trans. Syst. Man Cybern. 8 (6) (1978) 460–473.

[44] F. Liu, R.W. Picard, Periodicity, directionality, and randomness: wold features for image modeling and retrieval, IEEE Trans. Pattern Anal. Mach. Intell. 18 (7) (1996) 722–733.

[45] P. Brodatz, Textures, A Photographic Album for Artists & Designers, Dover, New York, NY, 1966.

[46] Y. Liu, X. Zhou, W.Y. Ma, Extraction of texture features from arbitrary-shaped regions for image retrieval, International Conference on Multimedia and Expo (ICME04), Taipei, June 2004, pp. 1891–1894.

[47] P.W. Huang, S.K. Dai, Image retrieval by texture similarity, Pattern Recognition 36 (2003) 665–679.

[48] R. Mehrotra, J.E. Gary, Similar-shape retrieval in shape data management, IEEE Comput. 28 (9) (1995) 57–62.

[49] F. Mokhtarian, S. Abbasi, Shape similarity retrieval under affine transforms, Pattern Recognition 35 (2002) 31–41.

[50] Y. Song, W. Wang, A. Zhang, Automatic annotation and retrieval of images, J. World Wide Web 6 (2) (2003) 209–231.

[51] A. Mojsilovic, B. Rogowitz, ISee: perceptual features for image library navigation, Proceedings of the SPIE, Human Vision and Electronic Imaging, vol. 4662, 2002, pp. 266–277.

[52] S.K. Chang, Q.Y. Shi, C.W. Yan, Iconic indexing by 2D string, IEEE Trans. Pattern Anal. Mach. Intell. 9 (3) (1987) 413–428.


Full Text

internal file

Sonstige Links