Accounting for Bias in the Evaluation of Creative Computational Systems: An Assessment of DARCI
Inhaltsverzeichnis
Reference
David Norton, Derrall Heath and Dan Ventura: Accounting for Bias in the Evaluation of Creative Computational Systems: An Assessment of DARCI. In: Computational Creativity 2015 ICCC 2015, 31-38.
DOI
Abstract
Recent investigations into the assessment and evaluation of “creative” systems in the field of computational creativity have disclosed several problems common to research within the field. We perform a practical evaluation of the latest iteration of the creative system, DARCI, attempting to address some of these problems using a specially designed, but generalizable, online human survey. Of note, we address the complications of evaluator bias that are present in all assessments of creativity. Using our evaluation, we show that within its narrow domain, DARCI is able to produce artifacts that are rated at least as favorably as human counter parts across five aspects of creativity. Further, these artifacts tend to be more surprising and perceived as more difficult to produce than those created by human artists.
Extended Abstract
Bibtex
@inproceedings{
author = {Norton, David and Heath, Derrall and Ventura, Dan},
title = {Accounting for Bias in the Evaluation of Creative Computational Systems: An Assessment of DARCI},
booktitle = {Proceedings of the Sixth International Conference on Computational Creativity},
series = {ICCC2015},
year = {2015},
month = {Jun},
location = {Park City, Utah, USA},
pages = {31-38},
url = {http://computationalcreativity.net/iccc2015/proceedings/2_2Norton.pdf http://de.evo-art.org/index.php?title=Accounting_for_Bias_in_the_Evaluation_of_Creative_Computational_Systems:_An_Assessment_of_DARCI },
publisher = {International Association for Computational Creativity},
keywords = {computational, creativity},
}
Used References
al Rifaie, M., and Bishop, M. 2012. Weak vs. strong computational creativity, computing, philosophy and the question of bio-machine hybrids. In Proceedings of the AISB Symposium on Computing and Philosophy.
Brown, O. 2014. Empirically grounding the evaluation of creative systems: Incorporating interaction design. In Proceedings of the International Conference on Computational Creativity.
Colton, S.; Pease, A.; Corneli, J.; Cook, M.; and Llano, T. 2014. Assessing progress in building autonomously creative systems. In Proceedings of the International Conference on Computational Creativity.
Colton, S. 2008. Creativity versus the perception of creativity in computational systems. Creative Intelligent Systems: Papers from the AAAI Spring Symposium 14–20.
Jordanous, A. 2012. A standardised procedure for evaluating creative systems: Computational creativity evaluation based on what it is to be creative. Cognitive Computation 4:246–279.
Jordanous, A. 2014. Stepping back to progress forwards: Setting standards for meta-evaluation of computational creativity. In Proceedings of the 5th International Conference on Computational Creativity.
King, I.; Ng, C. H.; and Sia, K. C. 2004. Distributed content-based visual information retrieval system on peerto- peer network. ACM Transactions on Information Systems 22(3):477–501.
Li, C., and Chen, T. 2009. Aesthetic visual quality assessment of paintings. IEEE Journal of Selected Topics in Signal Processing 3:236–252.
Machado, P.; Romero, J.; and Manaris, B. 2007. Experiments in computational aesthetics: An iterative approach to stylistic change in evolutionary art. In Romero, J., and Machado, P., eds., The Art of Artificial Evolution: A Handbook on Evolutionary Art and Music. Berlin: Springer. 381– 415.
Machajdik, J., and Hanbury, A. 2010. Affective image classification using features inspired by psychology and art theory. In Proceedings of the International Conference on Multimedia, 83–92.
Mart´ınez, H. P.; Yannakakis, G. N.; and Hallam, J. 2014. Dont classify ratings of affect; rank them! IEEE Transactions on Affective Computing 5:314–326.
Norton, D.; Heath, D.; and Ventura, D. 2013. Finding creativity in an artificial artist. Journal of Creative Behavior 47(2):106–124.
Norton, D.; Heath, D.; and Ventura, D. 2014. Autonomously managing competing objectives to improve the creation and curation of artifacts. In Proceedings of the 5th International Conference on Computational Creativity.
Norton, D.; Heath, D.; and Ventura, D. 2015. Annotating images with emotional adjectives using features that summarize local interest points. IEEE Transactions on Affective Computing, in submission.
Ou, L.-C.; Luo, M. R.;Woodcock, A.; and Wright, A. 2004. A study of colour emotion and colour preference. Part I: Colour emotions for single colours. Color Research and Application 29:232–240.
Ritchie, G. 2007. Some empirical criteria for attributing creativity to a computer program. Minds and Machines 17:67– 99.
Sivic, J., and Zisserman, A. 2003. Video Google: A text retrieval approach to object matching in videos. In Proceedings of the Ninth IEEE International Conference on Computer Vision, volume 2, 1470–1477.
Wang, W.-N.; Yu, Y.-L.; and Jiang, S.-M. 2006. Image retrieval by emotional semantics: A study of emotional space and feature extraction. In Proceedings of IEEE International Conference on Systems, Man, and Cybernetics, volume 4, 3534–3539.
Yannakakis, G. N., and Hallam, J. 2011. Rating vs. preference: A comparative study of self-reporting. Affective Computing and Intelligent Interaction 6974:437–446.
Links
Full Text
http://computationalcreativity.net/iccc2015/proceedings/2_2Norton.pdf
