Brain-Inspired Deep Networks for Image Aesthetics Assessment

Aus de_evolutionary_art_org
Wechseln zu: Navigation, Suche


Zhangyang Wang, Shiyu Chang, Florin Dolcos, Diane Beck, Ding Liu, and Thomas Huang: Brain-Inspired Deep Networks for Image Aesthetics Assessment. arXiv:1601.04155v2 [cs.CV] 15 Mar 2016



Image aesthetics assessment has been challenging due to its subjective nature. Inspired by the scientific advances in the human visual perception and neuroaesthetics, we design Brain-Inspired Deep Networks (BDN) for this task. BDN first learns attributes through the parallel supervised pathways, on a variety of selected feature dimensions. A high-level synthesis network is trained to associate and transform those attributes into the overall aesthetics rating. We then extend BDN to predicting the distribution of human ratings, since aesthetics ratings are often subjective. Another highlight is our first-of-its-kind study of label-preserving transformations in the context of aesthetics assessment, which leads to an effective data augmentation approach. Experimental results on the AVA dataset show that our biological inspired and task-specific BDN model gains significantly performance improvement, compared to other state-of-the-art models with the same or higher parameter capacity.

Extended Abstract


 author    = {Zhangyang Wang andFlorin Dolcos and Diane Beck and Shiyu Chang and Thomas S. Huang},
 title     = {Brain-Inspired Deep Networks for Image Aesthetics Assessment},
 journal   = {CoRR},
 volume    = {abs/1601.04155},
 year      = {2016},
 url       = {},
 timestamp = {Mon, 01 Feb 2016 15:36:05 +0100},
 biburl    = {},
 bibsource = {dblp computer science bibliography,}

Used References

1. Cheng, B., Ni, B., Yan, S., Tian, Q.: Learning to photograph. In: Proceedings of the international conference on Multimedia, ACM (2010) 291{300

2. Datta, R., Joshi, D., Li, J.,Wang, J.Z.: Studying aesthetics in photographic images using a computational approach. In: Computer Vision ECCV 2006. Springer (2006) 288-301

3. Ke, Y., Tang, X., Jing, F.: The design of high-level features for photo quality assessment. In: Computer Vision and Pattern Recognition, 2006 IEEE Computer Society Conference on. Volume 1., IEEE (2006) 419{426

4. Lu, X., Lin, Z., Jin, H., Yang, J., Wang, J.Z.: Rapid: Rating pictorial aesthetics using deep learning. In: Proceedings of the ACM International Conference on Multimedia, ACM (2014) 457-466

5. Murray, N., Marchesotti, L., Perronnin, F.: Ava: A large-scale database for aesthetic visual analysis. In: Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on, IEEE (2012) 2408{2415

6. Wu, O., Hu, W., Gao, J.: Learning to predict the perceived visual quality of photos. In: Computer Vision (ICCV), 2011 IEEE International Conference on, IEEE (2011) 225{232

7. Cela-Conde, C.J., Agnati, L., Huston, J.P., Mora, F., Nadal, M.: The neural foundations of aesthetic appreciation. Progress in neurobiology 94(1) (2011) 39-48

8. Chatterjee, A.: Neuroaesthetics: a coming of age story. Journal of Cognitive Neuroscience 23(1) (2011) 53-62

9. Bengio, Y.: Learning deep architectures for ai. Foundations and trends R in Ma- chine Learning 2(1) (2009) 1{127

10. Chatterjee, A.: Prospects for a cognitive neuroscience of visual aesthetics. Bulletin of Psychology and the Arts 4(2) (2003) 55-60

11. Bhattacharya, S., Sukthankar, R., Shah, M.: A framework for photo-quality assessment and enhancement based on visual aesthetics. In: Proceedings of the international conference on Multimedia, ACM (2010) 271-280

12. Dhar, S., Ordonez, V., Berg, T.L.: High level describable attributes for predicting aesthetics and interestingness. In: Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on, IEEE (2011) 1657-1664

13. Luo, W., Wang, X., Tang, X.: Content-based photo quality assessment. In: Computer Vision (ICCV), 2011 IEEE International Conference on, IEEE (2011) 2206-2213

14. Marchesotti, L., Perronnin, F., Larlus, D., Csurka, G.: Assessing the aesthetic quality of photographs using generic image descriptors. In: Computer Vision (ICCV) 2011 IEEE International Conference on, IEEE (2011) 1784-1791

15. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. International journal of computer vision 60(2) (2004) 91{110

16. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems. (2012) 1097-1105

17. Dong, Z., Shen, X., Li, H., Tian, X.: Photo Quality Assessment with DCNN that Understands Image Well. In: MultiMedia Modeling, Springer (2015) 524-535

18. Joshi, D., Datta, R., Fedorovskaya, E., Luong, Q.T., Wang, J.Z., Li, J., Luo, J.: Aesthetics and emotions in images. Signal Processing Magazine, IEEE 28(5) (2011) 94-115


Full Text

internal file

Sonstige Links