Creative Generation of 3D Objects with Deep Learning and Innovation Engines
Inhaltsverzeichnis
Reference
Joel Lehman, Sebastian Risi and Jeff Clune: Creative Generation of 3D Objects with Deep Learning and Innovation Engines. In: Computational Creativity 2016 ICCC 2016, 180-187
DOI
Abstract
Advances in supervised learning with deep neural networks have enabled robust classification in many real world domains. An interesting question is if such advances can also be leveraged effectively for computational creativity. One insight is that because evolutionary algorithms are free from strict requirements of mathematical smoothness, they can exploit powerful deep learning representations through arbitrary computational pipelines. In this way, deep networks trained on typical supervised tasks can be used as an ingredient in an evolutionary algorithm driven towards creativity. To highlight such potential, this paper creates novel 3D objects by leveraging feedback from a deep network trained only to recognize 2D images. This idea is tested by extending previous work with Innovation Engines, i.e. a principled combination of deep learning and evolutionary algorithms for computational creativity. The results of this automated process are interesting and recognizable 3D-printable objects, demonstrating the creative potential for combining evolutionary computation and deep learning in this way.
Extended Abstract
Bibtex
@inproceedings{
author = {Joel Lehman, Sebastian Risi and Jeff Clune},
title = {Creative Generation of 3D Objects with Deep Learning and Innovation Engines},
booktitle = {Proceedings of the Seventh International Conference on Computational Creativity},
series = {ICCC2016},
year = {2016},
month = {Jun-July},
location = {Paris, France},
pages = {180-187},
url = {http://www.computationalcreativity.net/iccc2016/wp-content/uploads/2016/01/Creative-Generation-of-3D-Objects-with-Deep-Learning-and-Innovation-Engines.pdf http://de.evo-art.org/index.php?title=Creative_Generation_of_3D_Objects_with_Deep_Learning_and_Innovation_Engines },
publisher = {Sony CSL Paris},
}
Used References
Baluja, S.; Pomerleau, D.; and Jochem, T. 1994. Towards automated artificial evolution for computer-generated images. Connection Science 6(2-3):325–354.
Bentley, P. J. 1996. Generic evolutionary design of solid objects using a genetic algorithm. Ph.D. Dissertation, The University of Huddersfield.
Boden, M. A. 1996. Dimensions of creativity. MIT Press. Clune, J., and Lipson, H. 2011. Evolving three-dimensional objects with a generative encoding inspired by developmental biology. Proceedings of the European Conference on Artificial Life, See http://EndlessForms.com 144–148.
Correia, J.; Machado, P.; Romero, J.; and Carballal, A. 2013. Evolving figurative images using expression-based evolutionary art. In Proceedings of the fourth International Conference on Computational Creativity (ICCC), 24–31.
Deng, J.; Dong, W.; Socher, R.; Li, L.-J.; Li, K.; and Fei- Fei, L. 2009. Imagenet: A large-scale hierarchical image database. In Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on, 248–255. IEEE.
Gatys, L. A.; Ecker, A. S.; and Bethge, M. 2015. A neural algorithm of artistic style. arXiv preprint arXiv:1508.06576.
Goodfellow, I.; Bengio, Y.; and Courville, A. 2016. Deep learning. Book in preparation for MIT Press.
He, K.; Zhang, X.; Ren, S.; and Sun, J. 2015. Deep residual learning for image recognition. arXiv preprint arXiv:1512.03385.
Hoover, A. K., and Stanley, K. O. 2009. Exploiting functional relationships in musical composition. Connection Science 21(2-3):227–251.
Horn, B.; Smith, G.; Masri, R.; and Stone, J. 2015. Visual information vases: Towards a framework for transmedia creative inspiration. In Proceedings of the Sixth International Conference on Computational Creativity June, 182.
Jia, Y.; Shelhamer, E.; Donahue, J.; Karayev, S.; Long, J.; Girshick, R. B.; Guadarrama, S.; and Darrell, T. 2014. Caffe: Convolutional architecture for fast feature embedding. In ACM Multimedia, volume 2, 4.
Laumanns, M.; Thiele, L.; Deb, K.; and Zitzler, E. 2002. Combining convergence and diversity in evolutionary multiobjective optimization. Evolutionary computation 10(3):263–282.
Lee, H.; Pham, P.; Largman, Y.; and Ng, A. Y. 2009. Unsupervised feature learning for audio classification using convolutional deep belief networks. In Advances in neural information processing systems, 1096–1104.
Lehman, J., and Stanley, K. O. 2011. Abandoning objectives: Evolution through the search for novelty alone. Evolutionary computation 19(2):189–223.
Liapis, A.; Martınez, H. P.; Togelius, J.; and Yannakakis, G. N. 2013. Transforming exploratory creativity with De- LeNoX. In Proceedings of the Fourth International Conference on Computational Creativity, 56–63. AAAI Press.
Lorensen, W. E., and Cline, H. E. 1987. Marching cubes: A high resolution 3D surface construction algorithm. In ACM siggraph computer graphics, volume 21, 163–169. ACM.
Machado, P.; Correia, J.; and Romero, J. 2012. Expressionbased evolution of faces. In Evolutionary and Biologically Inspired Music, Sound, Art and Design. Springer. 187–198.
Miller, G. A. 1995. Wordnet: a lexical database for english. Communications of the ACM 38(11):39–41.
Mordvintsev, A.; Olah, C.; and Tyka, M. 2015. Inceptionism: Going deeper into neural networks. Google Research Blog. Retrieved June 20.
Mouret, J.-B., and Clune, J. 2015. Illuminating search spaces by mapping elites. arXiv preprint arXiv:1504.04909.
Nguyen, A.; Yosinski, J.; and Clune, J. 2015a. Deep neural networks are easily fooled: High confidence predictions for unrecognizable images. In Computer Vision and Pattern Recognition (CVPR), 2015 IEEE Conference on, 427–436. IEEE.
Nguyen, A.; Yosinski, J.; and Clune, J. 2015b. Innovation engines: Automated creativity and improved stochastic optimization via deep learning. In Proceedings of the Genetic and Evolutionary Computation Conference.
Pugh, J. K.; Soros, L.; Szerlip, P. A.; and Stanley, K. O. 2015. Confronting the challenge of quality diversity. In Proc. of the Genetic and Evol. Comp. Conference.
Razavian, A. S.; Azizpour, H.; Sullivan, J.; and Carlsson, S. 2014. Cnn features off-the-shelf: an astounding baseline for recognition. In Computer Vision and Pattern Recognition Workshops (CVPRW), 2014 IEEE Conference on, 512–519. IEEE.
Saunders, R., and Gero, J. S. 2001. The digital clockwork muse: A computational model of aesthetic evolution. In Proceedings of the AISB, volume 1, 12–21.
Secretan, J.; Beato, N.; D’Ambrosio, D. B.; Rodriguez, A.; Campbell, A.; and Stanley, K. O. 2008. Picbreeder: Collaborative interactive evolution of images. Leonardo 41(1):98– 99.
Stanley, K. O. 2007. Compositional pattern producing networks: A novel abstraction of development. Genetic programming and evolvable machines 8(2):131–162.
Stiny, G., and Gips, J. 1971. Shape grammars and the generative specification of painting and sculpture. In IFIP Congress (2), volume 2.
Szegedy, C.; Liu, W.; Jia, Y.; Sermanet, P.; Reed, S.; Anguelov, D.; Erhan, D.; Vanhoucke, V.; and Rabinovich, A. 2015. Going deeper with convolutions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1–9.
Yosinski, J.; Clune, J.; Nguyen, A.; Fuchs, T.; and Lipson, H. 2015. Understanding neural networks through deep visualization. arXiv preprint arXiv:1506.06579.
Yumer, M. E.; Asente, P.; Mech, R.; and Kara, L. B. 2015. Procedural modeling using autoencoder networks. In Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology, 109–118. ACM.
Zhou, B.; Lapedriza, A.; Xiao, J.; Torralba, A.; and Oliva, A. 2014. Learning deep features for scene recognition using places database. In Advances in neural information processing systems, 487–495.
