Abstract
Metaphors are a powerful conceptual device to reason about human actions. As such, they have been heavily used in designing and describing human computer interaction. Since they can address scripted text, verbal expression, imaging, sound, and gestures, they can also be considered in the design and analysis of multimodal interfaces. In this paper we discuss the description and evaluation of the relations between metaphors and their implementation in human computer interaction with a focus on tangible user interfaces (TUIs), a form of multimodal interface. The objective of this paper is to define how metaphors appear in a tangible context in order to support their evaluation. Relying on matching entities and operations between the domain of interaction and the domain of the digital application, we propose a conceptual framework based on three components: a structured representation of the mappings holding between the metaphor source, the metaphor target, the interface and the digital system; a conceptual model for describing metaphorical TUIs; three relevant properties, coherence, coverage and compliance, which define at what extent the implementation of a metaphorical tangible interface matches the metaphor. The conceptual framework is then validated and applied on a tangible prototype in an educational application.
Similar content being viewed by others
Notes
We consider only differences relevant in terms of feedback to the user; e.g., a different computed result not returned to the user for subsequent interaction is not considered relevant in the metaphor analysis.
Cladograms are usually represented with the root at the bottom.
References
Alty JL, Knott RP (1999) Metaphor and human–computer interaction: a model based approach. In: Nehaniv CL (ed) Computation for metaphors, analogy, and agents. Springer, Berlin, pp 307–321
Alty JL, Knott RP, Anderson B, Smyth M (2000) A framework for engineering metaphor at the user interface. Interact Comput 13(2):301–322
Andersen P (1992) Computer semiotics. Scand J Inf Syst 4:3–30
Anderson B, Smyth M, Knott R, Bergan M, Bergan J, Alty J (1994) Minimising conceptual baggage: making choices about metaphor. In: People and computers IX, proceedings of HCI ’94. Cambridge University Press, New York, pp 179–194
Antle AN, Corness G, Bakker S, Droumeva M, Van Den Hoven E, Bevans A (2009) Designing to support reasoned imagination through embodied metaphor. In: Proceeding of the seventh ACM conference on creativity and cognition, CC 09, p 275
Averbukh V, Bakhterev M, Baydalin A, Ismagilov D, Trushenkova P (2007) Interface and visualization metaphors. In: Jacko J (ed) Human–computer interaction. Interaction platforms and techniques, lecture notes in computer science, vol 4551. Springer, Berlin, pp 13–22. doi:10.1007/978-3-540-73107-8-2
Averbukh V et al (2008) Searching and analysis of interface and visualization metaphors. In: Asai K (ed) Human computer interaction: new developments. InTech, New York
Bakker S, Antle A, Van Den Hoven E (2012) Embodied metaphors in tangible interaction design. Personal Ubiquitous Comput 16(4):433–449
Barr P (2003) User-interface metaphor in theory and practice. Master’s thesis, Victoria University of Wellington, Department of Mathematical and Computing Sciences
Barr P, Biddle R, Noble J (2002) A taxonomy of user interface metaphors. In: Proc. of SIGCHI-NZ Symposium On Computer-Human Interaction
Barr P, Biddle R, Noble J (2003) A semiotic model of user-interface metaphor. In: 6th International Workshop on Organisational Semiotics
Beaudouin-Lafon M (2000) Instrumental interaction. In: Proceedings of the SIGCHI conference on human factors in computing systems—CHI ’00, pp 446–453
Beaudouin-Lafon M (2004) Designing interaction, not interfaces. In: Proceedings of the working conference on advanced visual interfaces. ACM, New York, AVI ’04, pp 15–22
Black M (1954) Metaphor. Proc Aristot Soc N Ser 55:273–294
Blackwell AF (2006) The reification of metaphor as a design tool. ACM Trans Comput Hum Interact 13(4):490–530
Bortolaso C, Dubois E, Duranthon F, Bach C (2011) Co-design of interactive museographic exhibits: the MIME case study. In: Ciolfi L, Scott K, Barbieri S (eds) Re-thinking technology in museums. University of Limerick, Limerick, pp 37–48
Carroll JM, Mack RL (1985) Metaphor, computing systems, and active learning. Int J Man Mach Stud 22(1):39–57
Celentano A, Dubois E (2012) Metaphor modelling for tangible interfaces evaluation. In: Proceedings of the international working conference on advanced visual interfaces. ACM, New York, AVI ’12, pp 78–81
Coutaz J, Nigay L, Salber D, Blandford A, May J, Young R (1995) Four easy pieces for assessing the usability of multimodal interaction: the CARE properties. In: Nordby K, Helmersen PH, Gilmore DJ, Arnesen SA (eds) Proceedings of INTERACT ’95, IFIP TC13 interantional conference on human–computer interaction. Chapman & Hall, Lillehammer, pp 115–120
Coutrix C, Nigay L (2006) Mixed reality: a model of mixed interaction. In: Proceedings of the working conference on Advanced visual interfaces, AVI2006. ACM Press, New York, pp 43–50
Dubois E, Gray P (2007) A design-oriented information-flow refinement of the ASUR interaction model. In: Gulliksen J, Harning M, Palanque P, van der Veer G, Wesson J (eds) Proceedings of the international conference on engineering interactive systems EIS’07, vol 4940. Springer, Berlin, pp 465–482
Dubois E, Bortolaso C, Bach C, Duranthon F, Blanquer-Maumont A (2011) Design and evaluation of mixed interactive museographic exhibits. Int J Arts Technol 4(4):408–441
Fishkin K (2004) A taxonomy for and analysis of tangible interfaces. Pers Ubiquit Comput 8(5):347–358
Forceville CJ, Urios-Aparisi E (2009) Chapter 1. Introduction. In: Forceville CJ, Urios-Aparisi E (eds) Multimodal metaphor. Mouton de Gruyter, Berlin, pp 3–17
Gero J (2004) The situated function–behaviour–structure framework. Des Stud 25(4):373–391
Hartshorne C, Weiss P (eds) (1936) Collected papers of Charles Sanders Peirce, vol 1–6. Harvard University Press, Cambridge
Hennig W (1950) Grundzüge einer Theorie der Phylogenetischen Systematik. Deutscher Zentralverlag, Berlin
Hochmair H, Luttich K (2006) An analysis of the navigation metaphor-and why it works for the world wide web. Spatial Cognit Comput 6(3):235–278
Hornbaek K, Frokjaer E (2002) Evaluating user interfaces with metaphors of human thinking. In: Carbonell N, Stephanidis C (eds) Proceedings of the User interfaces for all 7th international conference on Universal access: theoretical perspectives, practice, and experience (ERCIM ’02). Springer, Berlin, pp 486–507
Hornecker E (2012) Beyond affordance. In: Proceedings of the sixth international conference on tangible, embedded and embodied interaction, TEI ’12, New York, New York, USA. ACM Press, New York, p 175
Hornecker E, Buur J (z2006) Getting a grip on tangible interaction: a framework on physical space and social interaction. In: Proc. CHI 2006. ACM Press, New York, pp 436–446
Hurtienne J (2009) Image schemas and design for intuitive use. Exploring new guidance for user interface design. PhD thesis, Technische Universitat Berlin
Hurtienne J, Blessing L (2007) Design for intuitive use—testing image schema theory for user interface design. In: ICED ’07, 16th international conference on engineering design, Paris, France, pp 1–12
Hurtienne J, Israel JH (2007) Image schemas and their metaphorical extensions: intuitive patterns for tangible interaction. In: Proceedings of the 1st international conference on Tangible and embedded interaction. ACM Press, New York, pp 127–134
Hurtienne J, Stößel C, Sturm C, Maus A, Rötting M, Langdon P, Clarkson J (2010) Physical gestures for abstract concepts: inclusive design with primary metaphors. Interact Comput 22(6):475–484
Indurkhya B (1986) Constrained semantic transference: a formal theory of metaphors. Synthese 68(3):515–551
Ishii H (2008) Tangible bits: beyond pixels. In: Proceedings of the 2nd international conference on Tangible and embedded interaction. ACM, New York, TEI ’08, pp xv–xxv
Ishii H (2008) The tangible user interface and its evolution. Commun ACM 51(6):32–36
Ishii H, Ben-Joseph E, Underkoffler J, Yeung L, Chak D, Kanji Z, Piper B (2002) Augmented urban planning workbench: overlaying drawings, physical models and digital simulation. In: ISMAR ’02, proceedings of the 1st international symposium on mixed and augmented reality. IEEE Computer Society, Washington, DC, pp 203–211
Jacob RJK, Girouard A, Hirshfield LM, Horn MS, Shaer O, Solovey ET, Zigelbaum J (2008) Reality-based interaction. In: Proceeding of the twenty sixth annual CHI conference on human factors in computing systems, CHI 08. ACM Press, New York, p 201
Johnson M (1987) The body in the mind. University of Chicago Press, Chicago
Jordà S, Geiger G, Alonso M, Kaltenbrunner M (2007) The reacTable: exploring the synergy between live music performance and tabletop tangible interfaces. In: Proceedings of the 1st international conference on tangible and embedded interaction. ACM, New York, pp 139–146
Jourde F, Laurillau Y, Morán AL, Nigay L (2008) Towards specifying multimodal collaborative user interfaces: a comparison of collaboration notations. In: Interactive systems. Design, specification, and verification, 15th international workshop, DSV-IS, Kingston, Canada, pp 281–286
Kay AC (1972) A personal computer for children of all ages. In: Proceedings of the ACM annual conference, vol 1. ACM, New York
Kintsch W (2000) Metaphor comprehension: a computational theory. Psychonom Bull Rev 7(2):257–266
Koleva B, Benford S, Ng KH, Rodden T (2003) A framework for tangible user interfaces. In: In workshop proceedings on real world user interfaces, mobile HCI conference 03, pp 257–264
Lakoff G (1987) Women, fire, and dangerous things: what categories reveal about the mind. University of Chicago Press, Chicago
Lakoff G (1993) The contemporary theory of metaphor. Metaphor and thought, pp 202–251
Lakoff G, Johnson M (1980) Metaphors we live by. University of Chicago Press, Chicago
Lamata P, Ali W, Cano A, Cornella J, Declerck J, Elle OJ, Freudenthal A, Furtado H, Kalkofen D, Naerum E, Samset E, Sánchez-Gonzalez P, Sánchez-Margallo FM, Schmalstieg D, Sette M, Stüdeli T, Sloten JV, Gómez EJ (2010) Augmented reality for minimally invasive surgery: overview and some recent advances. In: Maad S (ed) Augmented reality. InTech, Croatia
Macaranas A, Antle AN, Riecke BE (2012) Bridging the gap. In: Proceedings of the sixth international conference on tangible, embedded and embodied interaction, TEI ’12. ACM, New York, p 161
Mackay WE, Fayard AL, Frobert L, Médini L (1998) Reinventing the familiar: exploring an augmented reality design space for air traffic control. In: Proceedings of the SIGCHI conference on human factors in computing systems. ACM Press/Addison-Wesley Publishing Co., New York, CHI ’98, pp 558–565
Maquil V, Ras E, Zephir O (2011) Understanding the characteristics of metaphors in tangible user interfaces. Mensch Comput 2011:1–6
Maquil V, Zephir O, Ras E (2012) Creating metaphors for tangible user interfaces in collaborative urban planning: questions for designers and developers. In: Dugdale J, Masclet C, Grasso MA, Boujut JF, Hassanaly P (eds) From research to practice in the design of cooperative systems: results and open challenges. Springer, London, pp 137–151
Marcus A (1994) Managing metaphors for advanced user interfaces. In: Proceedings of the workshop on advanced visual interfaces. ACM, New York, AVI ’94, pp 12–18. http://doi.acm.org/10.1145/192309.192317
McGee D, Cohen PR (2001) Creating tangible interfaces by augmenting physical objects with multimodal language. In: Proceedings of the 6th international conference on Intelligent user interfaces. ACM, New York, IUI ’01, pp 113–119
McNerney TS (2004) From turtles to tangible programming bricks: explorations in physical language design. Pers Ubiquitous Comput 8(5):326–337. doi:10.1007/s00779-004-0295-6
Norman DA (2002) The design of everyday things. Basic Books, New York
Oppl S, Stary C (2011) Towards informed metaphor selection for TUIs. In: Proceedings of the 3rd ACM SIGCHI symposium on engineering interactive computing systems, EICS ’11. ACM, New York, pp 247–252
Perlman R (1976) Using computer technology to provide a creative learning environment for preschool children. Tech. rep., MIT AI Lab memo 360, Logo memo 24
Pittarello F, Stecca R (2010) Querying and navigating a database of images with the magical objects of the wizard Zurlino. In: IDC 2010, 9th international conference on interaction design and children. ACM Press, New York, pp 250–253
Pittarello F, Stecca R (2011) Mapping physical objects to digital functions: a tangible interface for querying and navigating a multimedia database. In: DEXA ’11, proceedings of the 22nd international workshop on database and expert systems applications. IEEE Computer Society, New York, pp 134–138
Poynor R (1995) The hand that rocks the cradle. ID the international design magazine, pp 60–65
Praggeljaz Group (2007) MIP: a method for identifying metaphorically used words in discourse. Metaphor Symbol 22(1):1–39
Price S, Jewitt C (2013) A multimodal approach to examining ’embodiment’ in tangible learning environments. In: Proceedings of the 7th international conference on tangible, embedded and embodied interaction. ACM, New York, TEI ’13, pp 43–50
Sajaniemi J, Stützle T (2007) Lightweight techniques for structural evaluation of animated metaphors. Interact Comput 19(4):457–471
Shaer O, Hornecker E (2010) Tangible user interfaces: past, present, and future directions. Found Trends Hum Comput Interact 3(1–2):1–137
Sickenius de Souza C (1993) The semiotic engineering of user interface languages. Int J Man Mach Stud 39(5):753–773
Steinhart E (2005) Generating & interpreting metaphors with NETMET. APA Newslett 4(2):3–7
Suzuki H, Kato H (1995) Interaction-level support for collaborative learning: algoblock—an open programming language. In: The first international conference on computer support for collaborative learning. L. Erlbaum Associates Inc., Hillsdale, CSCL ’95, pp 349–355. doi:10.3115/222020.222828
Svanaes D, Verplank W (2000) In search of metaphors for tangible user intefaces. In: Proceedings of DARE 2000 on designing augmented reality environments (DARE ’00), pp 121–129
Ullmer B, Ishii H (2000) Emerging frameworks for tangible user interfaces. IBM Syst J 39(3–4):915–931
Ullmer B, Ishii H, Jacob R (2005) Token+constraint systems for tangible interaction with digital information. ACM Trans Comput Hum Interact 12(1):81–118
Underkoffler J, Ishii H (1999) URP: a luminous-tangible workbench for urban planning and design. In: Proceedings of the SIGCHI conference on Human factors in computing systems, CHI ’99. ACM Press, New York, pp 386–393
Vaananen K, Schmidt J (1994) User interfaces for hypermedia: how to find good metaphors? In: Conference companion on Human factors in computing systems (CHI ’94), pp 263–264
Van Hees K, Engelen J (2013) Equivalent representations of multimodal user interfaces. Univers Access Inf Soc 12(4):339–368
Wagensberg J (2005) The “total” museum, a tool for social change. História, Ciências, Saúde Manguinhos 12(supplement):309–321
Zhou CL, Yang Y, Huang XX (2007) Computational mechanisms for metaphor in languages: a survey. J Comput Sci Technol 22(2):308–319
Acknowledgments
We thank Fabio Pittarello for fruitful discussions about his experiment, that we have used in Sect. 4.2 to introduce our framework. Figures 6 and 7 have been previously used in [22]. The authors wish to thank the Museum of Toulouse for providing the case study and the IJART journal, published by Inderscience, for allowing the reuse of this material.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Celentano, A., Dubois, E. Evaluating metaphor reification in tangible interfaces. J Multimodal User Interfaces 9, 231–252 (2015). https://doi.org/10.1007/s12193-015-0198-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12193-015-0198-z