Empathic painting: Interactive stylization using observed emotional state
Inhaltsverzeichnis
Reference
Shugrina, M., Betke, M., Collomosse, J.P.: Empathic painting: Interactive stylization using observed emotional state. In: Proc. ACM NPAR Sympos. (2006).
DOI
http://dx.doi.org/10.1145/1124728.1124744
Abstract
We present the "empathic painting" --- an interactive painterly rendering whose appearance adapts in real time to reflect the perceived emotional state of the viewer. The empathic painting is an experiment into the feasibility of using high level control parameters (namely, emotional state) to replace the plethora of low-level constraints users must typically set to affect the output of artistic rendering algorithms. We describe a suite of Computer Vision algorithms capable of recognising users' facial expressions through the detection of facial action units derived from the FACS scheme. Action units are mapped to vectors within a continuous 2D space representing emotional state, from which we in turn derive a continuous mapping to the style parameters of a simple but fast segmentation-based painterly rendering algorithm. The result is a digital canvas capable of smoothly varying its painterly style at approximately 4 frames per second, providing a novel user interactive experience using only commodity hardware.
Extended Abstract
Bibtex
Used References
Michael J. Black , Yaser Yacoob, Recognizing Facial Expressions in Image Sequences Using Local Parameterized Models of Image Motion, International Journal of Computer Vision, v.25 n.1, p.23-48, Oct. 1997 http://dx.doi.org/10.1023/A:1007977618277
Ian Buck , Adam Finkelstein , Charles Jacobs , Allison Klein , David H. Salesin , Joshua Seims , Richard Szeliski , Kentaro Toyama, Performance-driven hand-drawn animation, Proceedings of the 1st international symposium on Non-photorealistic animation and rendering, p.101-108, June 05-07, 2000, Annecy, France http://doi.acm.org/10.1145/340916.340929
Butler, A., Cleave, C. V., and Stirling, S. 1994. The Art Book. The Phaidon Press. ISBN: 0714829846.
Chau, M., and Betke, M. 2005. Real time eye tracking and blink detection with USB cameras. Tech. Rep. 2005--12, Boston University Computer Science.
Christopher M. Christoudias, Synergism in Low Level Vision, Proceedings of the 16 th International Conference on Pattern Recognition (ICPR'02) Volume 4, p.40150, August 11-15, 2002 http://dl.acm.org/citation.cfm?id=848599&CFID=588525319&CFTOKEN=29804931
J. P. Collomosse , P. M. Hall, Genetic paint: a search for salient paintings, Proceedings of the 3rd European conference on Applications of Evolutionary Computing, March 30-April 01, 2005, Lausanne, Switzerland http://dx.doi.org/10.1007/978-3-540-32003-6_44
John P. Collomosse , David Rowntree , Peter M. Hall, Stroke Surfaces: Temporally Coherent Artistic Animations from Video, IEEE Transactions on Visualization and Computer Graphics, v.11 n.5, p.540-549, September 2005 http://dx.doi.org/10.1109/TVCG.2005.85
DeCarlo, D., and Santella, A. 2002. Abstracted painterly renderings using eye-tracking data. In Proc. ACM SIGGRAPH, 769--776.
Duke, D., Barnard, P., Halper, N., and Mellin, M. 2003. Rendering and affect. In Proc. Eurographics, 359--368.
Ekman, P., and Friesen, W. 1978. Facial Action Coding System: A Technique for the Measurement of Facial Movement. Consulting Psychologists Press, Palo Alto, CA.
Bruce Gooch , Greg Coombe , Peter Shirley, Artistic Vision: painterly rendering using computer vision techniques, Proceedings of the 2nd international symposium on Non-photorealistic animation and rendering, June 03-05, 2002, Annecy, France http://doi.acm.org/10.1145/508530.508545
Paul Haeberli, Paint by numbers: abstract image representations, Proceedings of the 17th annual conference on Computer graphics and interactive techniques, p.207-214, September 1990, Dallas, TX, USA http://doi.acm.org/10.1145/97879.97902
IEEE Computer Graphics and Applications Staff, Almost Automatic Computer Painting, IEEE Computer Graphics and Applications, v.11 n.6, p.11-12, November 1991 http://dl.acm.org/citation.cfm?id=617699&CFID=588525319&CFTOKEN=29804931
Halper, N., Mellin, M., Herrmann, C. S., Linneweber, V., and Strothotte, T. 2003. Towards an understanding of the psychology of non-photorealistic rendering. In Proc. Workshop on Computational Visualistics, 67--78.
James Hays , Irfan Essa, Image and video based painterly animation, Proceedings of the 3rd international symposium on Non-photorealistic animation and rendering, June 07-09, 2004, Annecy, France http://doi.acm.org/10.1145/987657.987676
Henver, K. 1935. Experimental studies of the affective value of colors and lines. Journal of Applied Psychology, 385--398.
Aaron Hertzmann , Ken Perlin, Painterly rendering for video and interaction, Proceedings of the 1st international symposium on Non-photorealistic animation and rendering, p.7-12, June 05-07, 2000, Annecy, France http://doi.acm.org/10.1145/340916.340917
Aaron Hertzmann , Charles E. Jacobs , Nuria Oliver , Brian Curless , David H. Salesin, Image analogies, Proceedings of the 28th annual conference on Computer graphics and interactive techniques, p.327-340, August 2001 http://doi.acm.org/10.1145/383259.383295
Aaron Hertzmann, Painterly rendering with curved brush strokes of multiple sizes, Proceedings of the 25th annual conference on Computer graphics and interactive techniques, p.453-460, July 1998 http://doi.acm.org/10.1145/280814.280951
Aaron Hertzmann, Paint By Relaxation, Computer Graphics International 2001, p.47-54, July 03-06, 2001 http://dl.acm.org/citation.cfm?id=735238&CFID=588525319&CFTOKEN=29804931
Aaron Hertzmann, Fast paint texture, Proceedings of the 2nd international symposium on Non-photorealistic animation and rendering, June 03-05, 2002, Annecy, France http://doi.acm.org/10.1145/508530.508546
Kolliopoulos, A. 2005. Image segmentation for stylized non-photorealistic rendering and animation. Master's thesis, Univ. Toronto.
Kovacs, L., and Sziranyi, T. 2002. Creating video animations combining stochastic paintbrush transformation and motion detection. In Proc. 16thIntl. Conference on Pattern Recognition (ICPR), vol. II, 1090--1093.
Yan Li , Feng Yu , Ying-Qing Xu , Eric Chang , Heung-Yeung Shum, Speech-driven cartoon animation with emotions, Proceedings of the ninth ACM international conference on Multimedia, September 30-October 05, 2001, Ottawa, Canada http://doi.acm.org/10.1145/500141.500196
Peter Litwinowicz, Processing images and video for an impressionist effect, Proceedings of the 24th annual conference on Computer graphics and interactive techniques, p.407-414, August 1997 http://doi.acm.org/10.1145/258734.258893
Lundholm, H. 1921. The affective tone of lines: experimental researches. The Psychological Review 28, 60.
Mahnke, F. 1996. Color, Environment, and Human Response. Van Nostrand Reinhold.
Pentland, A., Moghaddam, B., and Starner, T. 1994. View-based and modular eigenspaces for face recognition. In Proc. Intl. Conf. on Comp. Vision and Pattern Recognition.
Pickford, R. W. 1972. Psychology and Visual Aesthetics.
Plutchik, R. 1980. A general psychoevolutionary theory of emotion. In Emotion: Theory, research, and experience, R. Plutchik and H. Kellerman, Eds. Academic press, Inc, 3--33.
Poffenberger, A. T., and Barrows, B. E. 1924. The feeling value of lines. Journal of Applied Psychology 8, 187--205.
Russel, J. A., and Fernández-Dols, J. M. 1997. The Psychology of Facial Expression. Cambridge University Press.
Russell, J. A. 1997. Reading emotion from and into faces: Resurrecting a dimensional-contextual perspective. In Russel and Fernández-Dols {Russel and Fernández-Dols 1997}, 295--320.
Anthony Santella , Doug DeCarlo, Visual interest and NPR: an evaluation and manifesto, Proceedings of the 3rd international symposium on Non-photorealistic animation and rendering, June 07-09, 2004, Annecy, France http://doi.acm.org/10.1145/987657.987669
Michio Shiraishi , Yasushi Yamaguchi, An algorithm for automatic painterly rendering based on local source image approximation, Proceedings of the 1st international symposium on Non-photorealistic animation and rendering, p.53-58, June 05-07, 2000, Annecy, France http://doi.acm.org/10.1145/340916.340923
Smith, C., and Scott, H. 1997. A componential approach to the meaning of facial expression. In The Psychology of Facial Expression, J. A. Russel and J. M. Fernández-Dols, Eds. Cambridge, 232--249.
Ying-li Tian , Takeo Kanade , Jeffrey F. Cohn, Recognizing Action Units for Facial Expression Analysis, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.23 n.2, p.97-115, February 2001 http://dx.doi.org/10.1109/34.908962
Treavett, S., and Chen, M. 1997. Statistical techniques for the automated synthesis of non-photorealistic images. In Proc. 15thEurographics UK Conference, 201--210.
Viola, P., and Jones, M. 2001. Rapid object detection using a boosted cascade of simple features. In Proc. Comp. Vision and Pattern Recognition (CVPR), vol. 1, 511--5128.
Jue Wang , Yingqing Xu , Heung-Yeung Shum , Michael F. Cohen, Video tooning, ACM SIGGRAPH 2004 Papers, August 08-12, 2004, Los Angeles, California http://doi.acm.org/10.1145/1186562.1015763
Wright, B., and Rainwater, L. 1962. The meaning of colour. Journal of General Psychology 67, 89--99.
Links
Full Text
http://www.cs.bu.edu/fac/betke/papers/old-ShugrinaBetkeCollomosse-2006.pdf
