This Special Issue is inspired by but not limited to the 2019 Edition of CHItaly, the biannual Conference of the Italian SIGCHI Chapter, an active community that brings together people working on the design, evaluation, and implementation of interactive computing systems for human use. The conference was held from 23 to 25 September 2019 in Padova, Italy. The Special Issue includes a selection of the best papers presented, but the call for papers also solicited high quality contributions from authors that did not attend the conference.

New interactive paradigms allow augmenting users’ physical possibilities, but also knowledge and engagement with different knowledge domains by enriching cognitive and emotional user experience. The aim of this Special Issue was to explore different applications of the new interaction paradigms, with special attention to sustainability, ability to user empowerment and to emotional interaction.

The finally accepted papers are six and can be divided into three groups.

  1. 1)

    New frontiers to valorize cultural heritage and enhance touristic experience:

The first paper in this group is “Locate Your Soundscape: Interacting with the Acoustic Environment” by Nicola Orio, Berardina De Carolis, and Francesco Liotard. The paper presents a platform, Locate your sound, for collecting and accessing environmental sounds that collectively build a soundscape, that can be defined as the auditory counterpart of a landscape. The platform was experimented through two different crowdsourcing initiatives, the first one focused on a city of Northern Italy, the latter one extended to the whole Italian territory during the COVID-19 lockdown. The creation of soundscapes was followed by an evaluation phase where volunteers could access the recordings through a web-based interface or through smart smartphones interacting with a NFC-enhanced map. Results show an improvement of the users’ awareness for the acoustic environment, that represents one important dimension of intangible cultural heritage, and are encouraging for the future development that will involve the participation to a country wide project on sound memories.

The second paper is “Leveraging Transmedia Storytelling to engage tourists in the understanding of destination’s local heritage” by Mara Dionisio and Valentina Nisi. Transmedia storytelling takes advantage of multiple delivery channels, and it is used in this paper to raise the awareness of tourists towards sustainable tourism practices, motivating the protection of nature, habitats and biodiversity. The connection with the local cultural and natural heritage is achieved through two interconnected components: a location-aware multimedia story where tourists are guided through the streets of the Madeira Island by a fictional character, Laura, and a hypermedia platform populated with locally connected testimonies. Overall, the evaluation reports promising results, in terms of emotional and cognitive involvement, and encourages the design of this kind of transmedia storytelling experiences.

  1. 2)

    Last technological tools to deal with healthcare and wellbeing:

Two papers in this special issue address this topic. The first paper is entitled “Phygital Interfaces for People with Intellectual Disability: an Exploratory Study at a Social Care Center.” It is authored by Mirko Gelsomini, Micol Spitale, and Franca Garzotto. Gelsomini and his colleagues describe a tool named Reflex to train cognitive skills in young adults with intellectual disabilities. This tool exploits the potential of so-called “phygital” interfaces, in which the users enter their input via physical objects that are semantically mapped into the digital content. Phygital interfaces’ potential for learning applications derives from exploiting the users’ sensorimotor abilities to manipulate concepts and abstract relations. The paper includes a case study where Reflex is used exploratorily in an actual social care center to interact with multimedia games (a Tangram task) via physical shapes. The exploratory results suggest that the phygital interface, although similar in task performance to all-physical or all-digital solutions, outweighs them in terms of participants’ preference. On the other hand, developing new games with Reflex might be more challenging to the personnel at the social care center than with traditional tools.

The second paper addressing healthcare and wellbeing is a work by Rita Francese, Michele Risi, Genoveffa Tortora, and Francesco Di Salle entitled “Thea: Empowering the Therapeutic Alliance of children with ASD by Multimedia Interaction.” The authors aim at developing a tool to improve the therapeutic alliance (TA) between caregivers and therapists of children with Autistic Spectrum Disorder. The paper describes the user studies leading to developing a multimedia application (“Thea”) that improves TA by facilitating sharing goals, plans, and progress among caregivers and therapists. The paper describes the design steps followed in collecting the user requirements, prototyping, and user experience evaluation with 31 caregivers. The paper ends with a helpful set of generalizable solutions to be adopted while developing a tool to support therapeutic alliance.

  1. 3)

    Approaches to game personalization and adaptation, in particular for children learning and ambient exploration experience:

The first paper in this group is “Reflection and Awareness in the Design Process - Children Ideating, Programming and Prototyping Smart Objects” by Rosella Gennari, Maristella Matera, Alessandra Melonio, Mehdi Rizvi, and Eftychia Roumelioti. The design of new technology with children has been widely investigated, also through workshops with children for designing novel IoT or smart objects. In this first phase, the main focus has been on technology. However, the research has moved from technological analysis towards a deeper analysis of possible cognitive gains that children can achieve in design workshops. The paper investigates on children’s gains in relation to their knowledge of design. It presents the outcomes of a structured design workshop with 27 children, from 11 to 14 years old. They are guided in ideating, programming and prototyping smart objects for their town park through a board-game with cards. The analysis of the gathered data suggests a positive influence of the workshops on children’s reflections and awareness of design.

The second paper is “DRAGON: Diversity Regulated Adaptive Generator Online” by Laura Anna Ripamonti, Federico Distefano, Marco Trubian, Dario Maggiorini, and Davide Gadia. The paper presents DRAGON (Diversity Regulated Adaptive Generator Online). It is an algorithm for facilitating the creation of “monster” archetypes for multiplayer games basing also on the players’ preferences. It is designed as a plugin, that has been implemented for one of the state-of-the-art game engines and tested with game developers. The work builds on approaches based on Procedural Content Generation (PCG) and their advantages. For instance, they support level designers and empower them in providing the possibility to produce personalized experiences for the players. In fact, the paper especially focuses on personalization aspects. The generation process exploits the genetic algorithm approach, that is of course adapted to guarantee enough flexibility to the game. The approach is well generalizable, since the archetypes produced by DRAGON can be employed for any game genre and setting.

The papers in this Special issue are only examples of new trends in the design of multimodal interaction supported by multimedia advanced artifacts. In the present technological context, the relations between the individual self, the society, the environment and technology are deeply changing. Information and Communication Technologies offer new opportunities to sense huge amounts of data from the entities mentioned above and to use such data to augment human knowledge, to influence behavior and to control our world in an active and proactive way. There are several questions arising from a technological scenario which changes with impressive speed. These questions include strategies and policies allowing the society and its various facets to take advantage of these new opportunities, but also the risks related to trespassing the ethical boundaries should regulate these strategies and policies. In particular, just because the ICT components that are progressively becoming a symbiotic part of our body and mind, it is necessary to keep the human in the loop through a responsible design. We hope that the ongoing research will always follow these principles.

We hope that the presented research can inspire the readers. Finally, we want to thank the authors for their contributions and the reviewers for their invaluable support.