Stepping Back to Progress Forwards: Setting Standards for Meta-Evaluation of Computational Creativity
Inhaltsverzeichnis
Reference
Anna Jordanous: Stepping Back to Progress Forwards: Setting Standards for Meta-Evaluation of Computational Creativity. In: Computational Creativity 2014 ICCC 2014, 129-136.
DOI
Abstract
There has been increasing attention paid to the ques- tion of how to evaluate the creativity of computational creativity systems. A number of different evaluation methods, strategies and approaches have been proposed recently, causing a shift in focus: which methodology should be used to evaluate creative systems? What are the pros and cons of using each method? In short: how can we evaluate the different creativity evaluation methodologies? To answer this question, five meta-evaluation criteria have been devised from cross-disciplinary research into good evaluative prac- tice. These five criteria are: correctness; usefulness; faithfulness as a model of creativity; usability of the methodology; generality. In this paper, the criteria are used to compare and contrast the performance of five various evaluation methods. Together, these meta- evaluation criteria help us explore the advantages and disadvantages of each creativity evaluation methodol- ogy, helping us develop the tools we have available to us as computational creativity researchers.
Extended Abstract
Bibtex
@inproceedings{
author = {Anna Jordanous},
title = {Stepping Back to Progress Forwards: Setting Standards for Meta-Evaluation of Computational Creativity},
booktitle = {Proceedings of the Fifth International Conference on Computational Creativity},
series = {ICCC2014},
year = {2014},
month = {Jun},
location = {Ljubljana, Slovenia},
pages = {129-136},
url = {http://computationalcreativity.net/iccc2014/wp-content/uploads/2014/06//8.3_Jordanous.pdf, http://de.evo-art.org/index.php?title=Stepping_Back_to_Progress_Forwards:_Setting_Standards_for_Meta-Evaluation_of_Computational_Creativity },
publisher = {International Association for Computational Creativity},
keywords = {computational, creativity},
}
Used References
Blanke, T. 2011. Using Situation Theory to evaluate XML retrieval. Dissertations in Database and Information Sys- tems. Heidelberg, Germany: IOS Press.
Colton, S.; Gow, J.; Torres, P.; and Cairns, P. 2010. Ex- periments in objet trouv ́e browsing. In Proceedings of the International Conference on Computational Creativity.
Colton, S.; Charnley, J.; and Pease, A. 2011. Computational Creativity Theory: The FACE and IDEA descriptive mod- els. In Proceedings of the 2nd International Conference on Computational Creativity, 90–95.
Colton, S. 2008. Creativity versus the perception of cre- ativity in computational systems. In Proceedings of AAAI Symposium on Creative Systems, 14–20.
Gillick, J.; Tang, K.; and Keller, R. M. 2010. Machine learn- ing of jazz grammars. Computer Music Journal 34(3):56– 66.
Jordanous, A., and Keller, B. 2014. What makes musical improvisation creative? Journal of Interdisciplinary Music Studies Forthcoming.
Jordanous, A. 2011. Evaluating evaluation: Assessing progress in computational creativity research. In Proceed- ings of the Second International Conference on Computa- tional Creativity (ICCC-11).
Jordanous, A. 2012a. Evaluating Computational Creativity: A Standardised Procedure for Evaluating Creative Systems and its Application. Ph.D. Dissertation, University of Sus- sex, Brighton, UK.
Jordanous, A. 2012b. A standardised procedure for evalu- ating creative systems: Computational creativity evaluation based on what it is to be creative. Cognitive Computation 4(3):246–279.
Kaufman, J. C. 2009. Creativity 101. The Psych 101 series. New York: Springer.
Kazai, G., and Lalmas, M. 2005. Notes on what to measure in INEX. In INEX 2005 Workshop on Element Retrieval Methodology.
Pease, A.; Winterstein, D.; and Colton, S. 2001. Evaluating machine creativity. In Proceedings of Workshop Program of ICCBR-Creative Systems: Approaches to Creativity in AI and Cognitive Science, 129–137.
Rhodes, M. 1961. An analysis of creativity. Phi Delta Kappan 42(7):305–310.
Ritchie, G. 2007. Some empirical criteria for attributing cre- ativity to a computer program. Minds and Machines 17:67– 99.
Sloman, A. 1978. The computer revolution in philosophy. Hassocks, Sussex: Harvester Press.
Thagard, P. 1988. Computational Philosophy of Science. Cambridge, MA: MIT Press.
Zhu, X.; Xu, Z.; and Khot, T. 2009. How creative is your writing? a linguistic creativity measure from computer sci- ence and cognitive psychology perspectives. In Proceedings of NAACL HLT Workshop on Computational Approaches to Linguistic Creativity (ACL), 87–93.
Links
Full Text
http://computationalcreativity.net/iccc2014/wp-content/uploads/2014/06//8.3_Jordanous.pdf
