Multimodality and Expressivity in Videogames

 

 

Nelson Zagalo*

*University of Aveiro, Portugal

 

 

ABSTRACT

This article addresses multimodality and expressivity in digital games, from the perspective of audiovisual communication, discussing essential critical dimensions regarding game literacy and experience. An overview on how we make meaning from audiovisual media is presented, to frame the focus on the multimodal cognitive nature of digital games. After comparing film and digital games, we discuss how and why the latter are rapidly becoming more relevant than television, discussing media learning differences between vicarious and enactive representations, and the creative need for children to access networks so as to interact in multimodal communication environments. 

Keywords: Digital games; multimodality; motivation; network; creativity.

 

 

Introduction

Videogames are still seen as dangerous media by a large number of societal players, and nothing demonstrates this fear better than having the World Health Organization (WHO) list the playing of these artefacts as hazardous for our mental health [1]. Even though some research has been able to identify certain problems with video games (Anderson, 2004; Stetina, Kothgassner, Lehenbauer, & Kryspin-Exner, 2011), several studies also demonstrate the reverse (Ferguson et al., 2008; Granic, Lobel, & Engels, 2014). A consensus is so far from being reached on the issue that the Diagnostic and Statistical Manual of Mental Disorders (DSM–5) decided not to list the behavior — playing games — as a mental problem in its latest edition, because of the lack of unquestionable evidence (Kuss, Griffiths, & Pontes, 2017).  Further regarding the WHO decision, a group of 36 researchers have presented a paper, in pre-print form, entitled — "A Weak Scientific Basis for Gaming Disorder: Let us err on the side of caution" (van Rooij, A. J., Ferguson, C. J., Colder Carras, M., Kardefelt-Winther, D., Shi, J., & Przybylski, 2018) — calling for the same cautionary attitude as that presented by the DSM, and conversely highlighting the possible stigmatization such actions can create in society. 

In addition, we have also seen intense debate taking place about the presence of children on the internet. In December 2017, Facebook launched a specific messaging tool — Messenger Kids — for children aged below 13 years old, the minimum age to have access to Facebook, and even after conducting thorough tests and studies involving parents [2], researchers, associations and organizations [3], and complying with their recommendations, its launch still led to strong protests calling for the application [4] to be shut down. When Facebook did not comply with these demands, they began asking parents to stop using the tool [5].

These two introductory stories serve to demonstrate the current situation regarding the use of communication technologies and digital games, and the levels of concern shown by our society towards them, especially when the focus is on children. Accordingly, this text aims to add to the debate, looking at the specificities of such artefacts, their design and multimodality capabilities, in order to raise literacy. Most of the fears and concerns arise through a lack of knowledge and a fear of the unknown, thus we hope to contribute to reducing such anxieties by offering an informed and scientifically-grounded discussion of the properties and capabilities of digital games. 

 

Multimedia, Film and Multimodality

Multimedia came to be the term used to define the integration of three key modalities — text, audio and moving images — by a computer-mediated means, in a very similar way to what film had already been doing, at least since 1927, the year when sound was added. The main difference between multimedia and film is that film is mixed by humans into a final and fixed form, while multimedia though also being manipulated by humans, is controlled by a computer which maintains independent access to each modality. In this sense, multimedia presented us with a new kind of communication mode: interactivity. Unlike with film, we could now interact with each modality — mixing, cutting, adding, and pushing the various media components — while experiencing the resulting multimedia artefact. This generated a need to understand not only what it would represent for its potential users, but also what could be accomplished with this new mode of representation.  

Thus, the 1980's saw the beginning of popular interest in the multimedia domain and what came to be defined as multimodal interaction (Bolt, 1980), gaining real traction in the 1990's (Coutaz, Salber, & Balbo, 1993) (Andersen, 1997) when the first International Conference on Multimodal Interaction was held in 1996 in Beijing [6].

Meanwhile, film theory evolved over the course of a century, emerging from philosophy and arts at a time when modernism was defining itself (Andrew, 1976); incorporating semiotics and linguistics (Metz, 1966) with the advent of structuralism; and then drawing on psychoanalysis (Bellour, Kuntzel, & Metz, 1975), feminism (Mulvey, 1975) and Marxism (Debord, 1967) through post-structuralism; eventually entering a period of all and nothing, with the coming of postmodernism (Friedberg, 1993); and finally arriving, at the end of the last century, at the cutting edge of the experimental method, first with psychology (Bordwell, 1985) and then with neuroscience (Plantinga & Smith, 1999).  

This serves to show that when research into multimodality, as an academic field, started to appear (Kress, 2000), announcing its interest in defining a field that lacked a theoretical framework (Kress, 2009), and  highlighting the problems of communication using different, non-textual modes (Bateman, 2008), such studies were not responding to a new problem. Film and multimedia had already been identified as being problematic in terms of representation. As far as text was concerned, there was the alphabet, and for music there were scores, yet nothing similar existed for film or multimedia communication. However, this was not because researchers had not tried to find a way of representing multimedia, with various studies attempting to do so following the multimodality approach — e.g. "Kineiconic mode" (Burn, 2016). The fact is that the complexity of audiovisual objects, their variability and infinite quality make it almost impossible to define a syntax to describe them, and consequently it is very difficult to attribute semantic values to them. 

 

Making meaning

Film studies have focused intensely on understanding how we make meaning from movies (Bordwell & Carroll, 1996). Such research takes inspiration from problematizing Plato and Aristotle's ideas about what representation should mean, a mimesis or diegesis, or even whether art should reduce itself to a simple imitation of reality, contradicting the possibility of abstract or anti-structural art forms. The discussion is long-standing, stagnating when it comes to both semiotics and cognitive approaches, the former seeking for an understanding of signs in film, while the latter deals with the mind and the way it constructs meaning (Bordwell, 1985; Boyd, 2010). From the perspective of the cognitive approach, understanding of the artefact, as far as meaning is concerned, is no different whether we use text, audio or moving pictures. Based on previous studies such as (Gallese & Goldman, 1998; Gibbs, 2006), Bergen (2012) argues that "we understand language by simulating in our minds what it would be like to experience the things that the language describes" (2012, p. 18). Therefore, it should not make any difference if we are simulating words, pictures or sounds inside our mind, as the brain will simply activate our memories and put them to work so as to build the correct simulation that will help us understand what is happening around us: "simulation is the creation of mental experiences of perception and action in the absence of their external manifestation" (2012, p. 18). 

However, there is a significant problem with our ability to simulate, and in the research carried out it has only worked well for more concrete facts and ideas (Casasanto & Boroditsky, 2008; Zhong & Leonardelli, 2008) — things that we can actually see, hear or do. This reminds us of the difference between text and film, whereby film can only express things it can show, and otherwise requires text, whether dialogue or some other form. This means that text is able to present ideas that cannot be sensed physically — such as guilt, pleasure, justice, and so on.  Bruner (1966) discussed these differences when he wanted to define representation modes, the way in which knowledge is encoded in memory, specifying three possibilities: "enactive, iconic and symbolic". In enactive representation, information is action-based, and things are learned by doing. The iconic representation is imagebased, built from concrete reality and stored to be searched or compared. The symbolic mode is code-based, referring to the way that language registers and encodes more abstract information. We could thus say, in terms of modes, that symbolic stands for text, iconic for visual, while enactive stands for the interactivity provided by multimedia, which brings us to the focus of this article, namely an attempt to define the distinct characteristics of multimedia and, more specifically, the representation of videogames. 

 

Games and Social Learning

Despite there being no syntax to define the language of film or videogames, we can work around their particular expressive capabilities, to better understand how each mode affects multimodality. The first step is to understand what separates these two media. Even if both qualify as belonging to the spectrum of audiovisual media, they bear strong differences. Videogames use moving images and sound, and can communicate through camera manipulations, even employing montage, the most radical expressive invention of film (Murch, 2001; Orpen, 2003), albeit with limitations (Zagalo, 2017). But what makes videogames a new medium stems from their multimedia capabilities, as mentioned above, the possibility of creating interaction between each mode and the user, such that we have come to define videogames as a specific kind of media: interactive media. 

We have been discussing representation modes, multimodal expressivity and multimedia capabilities but if we want to fully understand multimodal effects we also need to understand how information needs to be structured to be understood by humans. In 1985, Walter Fischer defined the human species as "homo narrans" (1989, p. 62), with Jerome Bruner subsequently characterizing "life as narrative" (1987), followed by Stephen Jay Gould's further definition of humans as "homo narrator" (1994). Since then a number of neuroscientific studies (Boyd, 2010; Damasio, 1994; Gottschall, 2012) have highlighted the importance of narrative for humans to generate understanding of the world. It has been said that it is easier for us not only to understand concepts and ideas in narrative form, but also to remember them. The narrative is nothing more than a specific, though very broad and simple means of organizing information, which also explains why we begin educating children by teaching them how to organize texts in narrative form. 

Being an information structure, narrative is also a social construct, following Walter Fisher, "all forms of human communication need to be seen fundamentally as stories" (1989, p. XI). This means that structure is not only present in artefacts but is socially used to create meaning. To connect narrative communication with the expressive capabilities of film and games, we bring into our discussion Albert Bandura and his "social cognitive learning" (1986) theory which divides the learning process into two different experiences: the "enactive" and the "vicarious". In the enactive experience, people learn by doing and experiencing, with consequences acting as information for the subjects. In the vicarious experience, people learn by observing the effects of consequences on other individuals. 

We can therefore say that films convey meaning to viewers by using narratives that help them experience reality vicariously. The question for videogames seems to be different. Even if videogames use narratives and tell stories, as in popular games like "Life is Strange" (2015) or "Mass Effect" (2007), they also use interaction expressively. In videogames, narratives are not only told, they are opened up for the players to choose, enact, and make changes to the stories being represented on the screen. If movies show these stories, videogames open a new approach to stories, giving agency to players, and putting them in the position of acting. The differences become clear, movies serve to produce vicarious experiences, while videogames produce enactive ones. 

To illustrate the difference, let me present an example [7] from the videogame "Life is Strange" (2015). In this videogame we play from the point of view of a teenage girl (Max) living through her existential angst, having only one friend (Chloe) with whom she is able to relate and open herself up to. The videogame uses narrative to structure the message, and essentially visual cinematics to depict the representation of the story. Even if the player acts upon the game, most of it is vicariously experienced, because players cannot really change its outcomes. However, at some specific points the game changes and permits some enacting, opening up choices for the player to decide on the fate of certain characters. One of these moments occurs at the end of the game, after Chloe has had an accident and is condemned to live the rest of her life in bed attached to a machine. Chloe goes and asks Max to disconnect the machine and help her die. This moment puts the vicarious aside, and obliges the player to actively do something, to determine what happens. The experience is poignant not because of the reward, but because no one wants to be put in that situation. We as players need to mentally elaborate an action based on our own beliefs. And in my case, a believer in individual freedom and the right to choose, I found that I could not let her die, in contradiction to my supposedly open beliefs. Even considering this was only a game, it certainly changed my perception since I was made to learn from the experience of being in that position, not merely theorizing about what I might do or watching someone else, but by feeling that it was genuinely up to me to decide upon another's fate. 

 

 

Maclntyre said "The difference between imaginary characters and real ones is not in the narrative form of what they do; it is in the degree of their authorship of that form and of their own deeds." (1981, p. 200). Narrative can occur in fictional worlds, and in "Life is Strange" characters are fictional, but the authorship of their actions does not belong to them, neither to the writer, but to us, the players. This changes completely the learning experience, because as defined by Bruner (1966), the mode of representation is no longer only iconic, but is enactive. In videogames we learn by doing, which is their most significant new expressive capability, and which also explains why, for the past 20 years, the whole education domain has been so interested in them. This is not a question of fashion, or just because children love to play games, but of different learning possibilities. 

 

Designing for Externalization

At present, videogames are one of the most celebrated forms of media, greatly enjoyed by both children and adults. However, it is not because the public love them that any new game will guarantee the interest and attention of players. As we have seen over the past thirty years, most educational games did not succeed in attracting player interest (Egenfeldt-Nielsen, 2007), and if we add to this equation the more than 500 new games that arrive daily at the App Store and Google Play, we see that the task for building games is not easy. Besides the importance of an interesting theme or story, games demand players to act, to become much more involved than when they watch a movie or a TV series. Thus, it is not enough to have a great story, some believable characters, impressive graphics, dynamic music or even an effective marketing strategy. In addition, games need to be able to feed and sustain players' motivation and encourage them to continue investing time in the game. A movie can last for two hours, but games can go on for 20 to 100 hours, so the core mechanics, or the set of actions that players need to undertake within the game need to be very well designed as, otherwise, they will leave before finishing it [8].

Consequently, creators have battled to find ways to maintain player interest during the game. This concern and associated research findings have pushed the boundaries of the medium, mainly through the concept known as gamification (Deterding, 2012), which refers to the use of game motivation techniques (ex. Points, Badges and 

Leaderboards) to help raise interest in other domains. Likewise, social media has seen an explosion of interest in gaming practices to maintain interest and focus within each social app, and the techniques have become so advanced that there is widespread fear of their capability to generate addiction (Alter, 2017). Different movements are now pushing for a level of control over these social tools, warning that they potentially rob people of experiencing regular life (Turkle, 2015). Such public and academic concerns are understandable, but as we explained at the beginning of this article, tools and games need to be seen in perspective. These are no chemical drugs, they are media, and we know from our history that the appearance of every new medium, from the comics to the television, has generated fears and concerns. Even if we accept that some reports have demonstrated that certain people have suffered from addiction to such games, what we should investigate is whether the games were not being used as coping mechanisms for other real-life issues. 

Having presented possible problems, we need now to discuss the potential gains, or the significance of digital game design practice, how it works, and the effects and implications for children. Essentially, vicarious learning has a very important role to play when it can effectively provide an interactive domain and lead children towards an understanding of the world by enabling them to enact potential future experiences, and learn by performing them while in a safe environment (Gee, 2008). Research on the use and gains of simulation systems have presented high rates of success in such different areas such as: aviation (Salas, Bowers, & Rhodenizer, 1998), nursing (Kaakinen & Arwood, 2009), military (Smith, 2010), among others. We know that young children, the focus of our attention here, are clearly not prepared for professional simulations, but that does not mean that they do not also have the potential need to enact certain interactive situations, considering, for example, the interaction requests aimed at children from the toy industry. To focus our argument, the relevance of simulation for young children can be determined around the idea of creating their own identity, to help them define their tastes, and, moreover, to open spaces for self-externalization, imagination and creativity (Vygotsky, 1930). Creative play has been approached by some digital games, and we know that children are highly creative by nature, although not always able to externalize their creativity, because it is not enough to give them blank pages and a lot of colors to play with. On the other hand, if rules are too prescriptive, explaining what they should do and how in order to attain some kind of gaming goal, they could hinder more than help by limiting their freedom. 

 

 

In the past, certain digital games like "Club Penguin" have been designed specifically for young children, pushing social boundaries and literacy (Marsh, 2014) Nowadays, Toca Boca studio is well-known because it aims specifically to achieve the idea of developing spaces for children's creativity (Daniels, 2017; Kopelke & Gower, 2017; Sari,

Takacs, & Bus, 2017). In their games, which are very diverse (e.g. "Toca Life", "Toca Lab", "Toca Hair", "Toca Dance", "Toca Kitchen"), the approach to interaction is defined by openness and freedom, avoiding rules or narrow objectives. Toca Boca privileges the intrinsic motivation of kids, presenting them with themes and ideas, opening paths for them to choose what to follow. There are no carrots to push children into playing, although the design appreciates what information to present and how to do so in order to attract and build their motivation for interaction and express themselves through the games. This specificity with regard to how such engagement works is what will be discussed in the next point. 

 

Motivation and Peers

To understand how digital games like Toca Boca work, we first need to look at the basic principles of human motivation. In the past, it was believed that to motivate people, or children, basic behavioral tools, like punishments and rewards, were enough. Even today, we still believe in this, and if we look at the mainstream digital games, the great majority use such tools to encourage people to play — giving or taking away "points"; attributing "badges" for achievements; or putting names in "leaderboards", to push people into competition (Hamari, Koivisto, & Sarsa, 2014). However, in the 1980s Deci and Ryan (1985) proposed a new approach, called "self-determination theory". Their experiments [9] and the popular communication of their theories (Pink, 2010) opened the possibility for new design approaches to human engagement. 

This new motivational theory arose to counter the behavioral approach, arguing that there are two different kinds of motivation, namely intrinsic and extrinsic. Essentially, extrinsic motivation is created with behavioral tools, thus engaging subjects with external objectives. Intrinsic motivation, meanwhile, refers to conditions that cause people to act based on their own, internal, objectives.  Deci and Ryan thus asserted the need for three key elements: "autonomy", "competence" and "relatedness". Accordingly, to attract and maintain human interest, we need first to develop spaces where individuals have some autonomy to choose and decide upon those spaces. Second, such spaces must require skills that are in an attainable range for the subjects, since if they are too easy or too difficult, people will lose interest. And third, the actions must not be individualized but should relate to someone, whether a friend or a person with the same interests (Deci and Ryan, 1985). 

In a recent research project [10] on developing digital games for the kindergarten, children aged below 6/8 years old, we found that games like those created by Toca Boca studio are very good at supporting levels of autonomy and competence. In table 1 we can see the most significant mechanics being designed in games for ages below 6/8, and which are being privileged by each game studio. Since research has already been conducted into the way these mechanics promote each of the elements of self-determination (Zagalo, N., Carvalho, A. A., & Araújo, 2016), we thus focus here on connecting each game with each design element to find out what motivation elements were being privileged by each game. As we can see in table 1, the games at the top, present opportunities for children to choose, build, and evolve at their own rhythm, while those at the bottom are more concerned about guaranteeing [11] that specific paths are followed and obstacles tackled, as well as quantifying the experience and success. The top ones are therefore designed to encourage autonomy, developing competences in wider contexts (not presenting specific content as obligatory, but opening different possibilities) providing more scaffolding to learning (Wood, Bruner, & Ross, 1976). On the other hand, the games at the bottom of the table encourage the development of competences through competition using extrinsic motivation elements, employing rules and obstacles, as well as using quest and conquer quantifications. 

In doing this research, we came to find that, even if a part of the games were already using self-determinacy elements, such as autonomy and competence, they lacked attempts to exploit the third intrinsic motivational element, "relatedness" (Zagalo & Laranjeiro, 2018). Researching the designers' goals, through interviews and views expressed by the companies, we found that relatedness was not being ignored, although companies did not feel very comfortable in designing for it. As outlined in the stories opening this article, our society does have some problems accepting children accessing online networks, and this reduces the possibility for these games to encourage intrinsic relatedness.

In the world of digital games there are so many opportunities to work with networks and yet also keep children safe. To illustrate this, we can use the example of the digital game "Journey" (2012), in which we play as an exotic character that needs to find the mountain where the sun is shining brightest, so as to be liberated. To do so, the player has to go through an immense array of obstacles, but at some points in the game, certain actions need to be performed by at least two characters, and so if you are connected to the internet, the game will connect you with someone also playing the game at that moment. While playing, we know the other character is also a player like us, a human being with feelings. We think about what we are doing in the game, how we can do it, but we also work to empathize and engage with the other player. Not because we need her/him to play, but because she/he is someone very like us, and even if we cannot talk with each other, we can see what each of us is doing all the time, in order to keep progressing forward in the game. We need to collaborate, but also maintain a high level of freedom to roam and do whatever we want. Therefore, when we arrive at the final level, and we are both able, simultaneously, to cross the mountain together, there is a kind of inner feeling of satisfaction that is almost impossible to describe. It is the achievement of finishing the game, but also the feeling of having done it because of the help and presence of the other person, and so the opportunity to share with someone who we have never met, makes us develop a sort of feeling of belonging, of gratitude. Creating spaces for relatedness must not, therefore, be seen as merely providing spaces of conversation, as we could see in "Penguin Club" (2005).  

Another relevant example could be extracted from "Minecraft" (2009), a hugely successful game with almost any age, with the Education version being targeted at ages as low as: 3 to 5 and 6 to 7 years old [21]. Ever since it began, this game knew how to motivate players, presenting plenty of freedom, "autonomy", structuring levels of required "skill" that would serve almost any player, and pushing the right buttons of intrinsic motivation. However, the game would never have had the success it achieved if it did not present a space for children to play and create together. The research done around the game "Minecraft" presents the main gain of playing the game as "fostering creativity" (Mavoa, Carter, & Gibbs, 2017). "Minecraft" connects players in order to collaborate, instead of competing. The goals are to build, and children can build bigger if they unite with others. This creates a very strong layer of additional motivation to continue playing the game, explaining in part why "Minecraft" became the second most sold game in history [22]. Furthermore, "Minecraft" has been the digital game most adopted in schools throughout the world, for the teaching of topics as diverse as coding, chemistry, physics, math, environment, electronics, and arts, among others. Mojang studio in collaboration with Microsoft have even developed a specific version of the game, "Minecraft: Education Edition" [25], maintaining the opportunity for children to share their work, but also providing a collaborative layer for teachers, encouraging them to share their lessons and ideas for teaching.

 

 

 

Literacy gains and educational uses

Throughout the discussion presented in this article we have seen the new possibilities opened up by multimedia and games, their integration of different modes, and the offering of a new mode, namely interactivity, which, as Bruner argued, opens new spaces of representation for enactive learning modes. In addition, we saw how games harness motivation to elicit enactive behaviors from players, the actions requested by the design mechanics that guarantee engagement and interest in going on playing games beyond the simplistic "carrot and stick" approaches. 

Enactive learning approaches, provided by the multimodality of videogames, open up spaces for children to be more active and less passive, and so develop a more profound understanding of representations. It makes possible a far more personalized view of the world, less defined by other people. Enacting actions demand a critical approach, and an in-depth understanding of context instead of just believing in what is being portrayed in vicarious modes. In terms of literacy, the main gains occur with creation of knowledge that becomes more syncretic and less unidirectional, requiring children to look for information outside of the realm of representations, and then integrate everything within so as to respond to the choices required of them. This happens mostly through searching the internet for more information about games, about stories represented in games, and asking adults (parents) for cooperation. Another realm is that of interaction with other children, and the exchange of experiences, and this area has been highly developed in recent years with the possibility of sharing playing experiences on YouTube, which explains why some of these shared plays have gathered millions of views [23]. Children look into the world around them for cues and explanations to then integrate into their world view and thus be able to perform within games. This turns enactive learning into tangible learning, providing the most important perspective on using games in education, which is to see them more as complementary devices and less as information containers and directing tools. 

To support the idea of tangible learning being stimulated by the multimodality of videogames, we have indicated the importance of motivation to play. One of the main problems of educational games over the past 30 years has been the lack of interest in what such games demand from children. Children understand when they are being directed and tend to reject it, not because the content of educational games is related to school matters, but because such games destroy the essential autonomy and liberty that videogames have alternatively offered to players. This autonomy is fundamental to maintain interest in games and supports the idea that educational games should work more effectively for tangential learning. A vital part of enactive learning is that for children to completely understand the realm of the game they are playing, they need to search for more context about what the game is about. Of course, competence and scaffolding are still highly relevant in games (Gee, 2004), but they have been well implemented and to some extent provide motivation, although not as much as they could, considering what we have seen of the lack of success achieved by educational games. 

Finally, and in support of the third vector in the self-determination approach, that is, relatedness, we have seen above how children need to interact with the world around the game to form a more concrete idea about what they are playing. This is new when compared to other media, like film or television, but is not so new when compared to physical games or just the act of playing. As the director of the National Institute for Play, Stuart Brown (2009) said, play impacts "intelligence" and "socialization", with most acts of play occurring with others, thus stimulating "social bonding" (Fredrickson & Branigan, 2005). This is why we have been defending the need to push for a more networked playing environment. Having children playing alone versus the computer diminishes the possibilities of them growing, transforming their inner self and evolving through learning about the surrounding world.  

Intrinsic motivation, as argued by the self-determination theory, is not only a question of motivating humans, but above all a question of self-realization, creation of identity and understanding others. Education should continue to look into how games help to encourage these learning objectives, but instead of wanting games to act as textbooks, educators should look for games to act as challengers, to help children look beyond their own world, their classroom, and their social structure, and, instead, to look for differences, comparisons, and opportunities to learn about themselves. 

 

Conclusions

Since digital games are interactive audiovisual experiences, everything considered in regard to the multimodality of movies should, therefore, also be applied here. However, the center of the cognitive effort and learning distinction that challenges videogame players goes beyond the cognitive interpretation needed to understand audiovisual content. In order to fully comprehend and learn from the content encapsulated in digital games, children need to perform, to act upon the simulated environment, take decisions and thus become part of the experience. 

We have seen that to understand reality, we use and require stories, which are conveyed through narrative structures. To make us aware of others, their reasoning and feelings, narratives employ empathy, so as to effectively put the recipient of the story in the characters' shoes. All this can then be mediated through different artefacts — from books to movies, theatre plays, audio dramas, television programs, journal series, and of course digital games. Digital games draw on all the tools used by movies to tell stories, but because they have a computer running the entire program all the time, they can open different access points into these stories, which create interactive moments, providing the opportunity for the particular features of simulation. Therefore, digital games, unlike other media, offer modes which do more than just feed players with vicarious experiences, but allow them to experience genuine enactive learning moments, albeit in simulated environments, and engage with their principal mode, namely interactivity. 

Consequently, we should not be rejecting digital games just because they are different and new. Moreover, as we have demonstrated throughout this article, digital games can help in the learning process, and they can certainly be part of our children's lives. The essential motivational potential that they offer can be of great value for many children out there who do not have the necessary strength to easily express themselves. We need to consider the different possible ways of enabling children to go online and exchange ideas and expressions but, especially, to collaborate and cooperate to create better places. We cannot really just hide ourselves behind a brick wall out of fear of the unknown.  

 

Acknowledgements

This work was co-financed by the European Union through the European Regional Development Fund, the 2020 Center Program, under the project "Learn 21: Game-based m-learning for children in kindergarten" (CENTRO010247-FEDER-009828). 

 

References

Alter, A. (2017). Irresistible: The Rise of Addictive Technology and the Business of Keeping Us Hooked. New York, USA: Penguin Press.         [ Links ]

Andersen, R. A. (1997). Multimodal integration for the representation of space in the posterior parietal cortex. Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences, 352(1360), 1421– 1428. https://doi.org/10.1098/rstb.1997.0128         [ Links ]

Anderson, C. A. (2004). An update on the effects of playing violent video games. Journal of Adolescence, 27(1), 113–122. https://doi.org/10.1016/j.adolescence.2003.10.009         [ Links ]

Andrew, J. D. (1976). The major film theories: An introduction. London, UK: Oxford University Press.         [ Links ]

Bandura, A. (1986). Social foundations of thought and action: a social cognitive theory. New Jersey: Prentice-Hall, Pearson.         [ Links ]

Bateman, J. A. (2008). Multimodality and Genre: A Foundation for the Systematic Analysis of Multimodal Documents. London, UK: Palgrave Macmillan. https://doi.org/10.1057/9780230582323        [ Links ]

Bellour, R., Kuntzel, T., & Metz, C. (1975). Psychanalyse et cinéma, Communications, 23. Paris: Editions Seuil.         [ Links ]

Bergen, B. (2012). Louder than words: The new science of how the mind makes meaning. New York: Basic Books.         [ Links ]

Bolt, R. (1980). "Put-that-there": Voice and Gesture at the Graphics Interface. In SIGGRAPH '80 Proceedings of the 7th annual conference on Computer graphics and interactive techniques (pp. 262–270). Seattle, Washington, USA. https://doi.org/10.1145/800250.807503

Bordwell, D. (1985). Narration in the Fiction Film. Wisconsin: University of Wisconsin Press.

Bordwell, D., & Carroll, N. (1996). Post-Theory: Reconstructing Film Studies. Wisconsin: University of Wisconsin Press.

Boyd, B. (2010). On the Origin of Stories. Cambridge, Massachusetts: Harvard University Press.

Brown, S. L. (2009). Play: How it shapes the brain, opens the imagination, and invigorates the soul. New York: Avery - Penguin Group.         [ Links ]

Bruner, J. (1966). Toward a Theory of Instruction. Cambridge, Massachusetts: Harvard University Press.

Bruner, J. (1987). Life as Narrative. Social Research, 54(1), 11–32.         [ Links ]

Burn, A. (2016). Making machinima: animation, games, and multimodal participation in the media arts. Learning, Media and Technology, 41(2), 310–329. https://doi.org/10.1080/17439884.2015.1107096         [ Links ]

Casasanto, D., & Boroditsky, L. (2008). Time in the mind: Using space to think about time. Cognition, 106(2), 579– 593. https://doi.org/10.1016/j.cognition.2007.03.004         [ Links ]

Coutaz, J., Salber, D., & Balbo, S. (1993). Towards automatic evaluation of multimodal user interfaces. Knowledge Based Systems, 6(4), 267–274. https://doi.org/10.1016/0950-7051(93)90018-O         [ Links ]

Damasio, A. (1994). Descartes' Error: Emotion, Reason and the Human Brain. New York, New York, USA: Putnam Publishing.

Daniels, K. (2017). Children's Engagement with iPads in Early Years Classrooms: Exploring Peer Cultures and Transforming Practices. In B. C., M. G., S. A., & W. M. (Eds.), The Case of the iPad (pp. 195–210). Singapore: Springer Singapore https://doi.org/10.1007/978-981-10-4364-2_12         [ Links ]

Debord, G. (1967). La Société du Spectacle. Paris, France: Gallimard.         [ Links ]

Deci, E. L., & Ryan, R. M. (1985). Intrinsic Motivation and Self-Determination in Human Behavior. Boston, MA: Springer US. https://doi.org/10.1007/978-1-4899-2271-7

Deterding, S. (2012). Gamification. Interactions, 19(4), 14. https://doi.org/10.1145/2212877.2212883        [ Links ]

Egenfeldt-Nielsen, S. (2007). Educational potential of computer games. Journal of Educational Multimedia and Hypermedia, 16(3), 263–281.         [ Links ]

Ferguson, C. J., Rueda, S. M., Cruz, A. M., Ferguson, D. E., Fritz, S., & Smith, S. M. (2008). Violent Video Games and Aggression. Criminal Justice and Behavior, 35(3), 311–332. https://doi.org/10.1177/0093854807311719        [ Links ]

Fisher, W. R. (1989). Human Communication as Narration. Columbia, South Carolina: University of South Carolina Press.

Fredrickson, B. L., & Branigan, C. (2005). Positive emotions broaden the scope of attention and thought-action repertoires. Cognition and Emotion, 19(3), 313–332. https://doi.org/10.1080/02699930441000238         [ Links ]

Friedberg, A. (1993). Window shopping: Cinema and the postmodern. Berkeley: University of California Press.         [ Links ]

Gallese, V., & Goldman, A. (1998). Mirror neurons and the mind-reading. Trens in Cognitive Sciences, 2(12), 493– 501. https://doi.org/10.1016/S1364-6613(98)01262-5        [ Links ]

Gee, J. P. (2003). What Video Games Have to Teach Us about Learning and Literacy. New York, New York, USA: Palgrave Macmillan.

Gee, J. P. (2008). Video games and embodiment. Games and Culture, 3(3–4), 253–263. https://doi.org/10.1177/1555412008317309        [ Links ]

Gibbs, R. W. (2006). Metaphor interpretation as embodied simulation. Mind and Language, 21(3), 434-458. https://doi.org/10.1111/j.1468-0017.2006.00285.x        [ Links ]

Gottschall, J. (2012). The Storytelling Animal: How Stories Make Us Human. Boston, MA: Houghton Mifflin Harcourt.

Gould, S. J. (1994). So Near and Yet So Far. The New York Review of Books. Retrieved from https://www.nybooks.com/articles/1994/10/20/so-near-and-yet-so-far/        [ Links ]

Granic, I., Lobel, A., & Engels, R. C. M. E. (2014). The benefits of playing video games. American Psychologist, 69(1), 66–78. https://doi.org/10.1037/a0034857         [ Links ]

Hamari, J., Koivisto, J., & Sarsa, H. (2014). Does Gamification Work? -- A Literature Review of Empirical Studies on Gamification. In 2014 47th Hawaii International Conference on System Sciences (pp. 3025–3034). IEEE. https://doi.org/10.1109/HICSS.2014.377

Kaakinen, J., & Arwood, E. (2009). Systematic Review of Nursing Simulation Literature for Use of Learning Theory. International Journal of Nursing Education Scholarship, 6(1), https://doi.org/10.2202/1548923X.1688         [ Links ]

Kopelke, K., & Gower, S. (2017). Building digital construction zones for young learners. Educating Young Children: Learning and Teaching in the Early Childhood Years, 23(1), 46–48.         [ Links ]

Kress, G. (2000). Multimodality. In B. Cope & M. Kalantzis (Eds.), Multiliteracies: Literacy learning and the design of social futures (pp. 179–200). London: Routledge.         [ Links ]

Kress, G. (2009). Multimodality: A social semiotic approach to contemporary communication. London: Routledge. https://doi.org/10.4324/9780203970034         [ Links ]

Kuss, D. J., Griffiths, M. D., & Pontes, H. M. (2017). DSM-5 diagnosis of Internet Gaming Disorder: Some ways forward in overcoming issues and concerns in the gaming studies field: Response to the commentaries. Journal of Behavioral Addictions, 6(2), 133–141. https://doi.org/10.1556/2006.6.2017.032         [ Links ]

MacIntyre, A. (1981). After Virtue: A Study in Moral Theory. Notre Dame, Indiana: University of Notre Dame Press.

Marsh, J. (2014). Purposes for literacy in children's use of the online virtual world Club Penguin. Journal of Research in Reading, 37(2), 179–195. https://doi.org/10.1111/j.1467-9817.2012.01530.x         [ Links ]

Mavoa, J., Carter, M., & Gibbs, M. (2017). Beyond Addiction. In B. Schouten, P. Markopoulos, & Z. Toups (Eds.), Proceedings of the Annual Symposium on Computer-Human Interaction in Play (pp. 171–181). Amsterdam, Netherlands: ACM Digital Press. https://doi.org/10.1145/3116595.3116638         [ Links ]

Metz, C. (1966). Les sémiotiques ou sémies. Communications, 7(1), 146–157. https://doi.org/10.3406/comm.1966.1102         [ Links ]

Mulvey, L. (1975). Visual pleasure and narrative cinema. Screen, 16(3), 6–18.         [ Links ]

Murch, W. (2001). In the blink of an eye: A perspective on film editing. Hollywood, CA: Silman-James Press.

Orpen, V. (2003). Film Editing: The Art of the Expressive. New York, New York, USA: Wallflower Press.

Pink, D. (2009). Drive: The Surprising Truth About What Motivates Us. New York, New York, USA: Riverhead Books.

Plantinga, C. R., & Smith, G. M. (1999). Passionate views: film, cognition and emotion. Baltimore, USA: Johns Hopkins University Press.         [ Links ]

Salas, E., Bowers, C. A., & Rhodenizer, L. (1998). It Is Not How Much You Have but How You Use It: Toward a Rational Use of Simulation to Support Aviation Training. The International Journal of Aviation Psychology, 8(3), 197–208. https://doi.org/10.1207/s15327108ijap0803_2        [ Links ]

Sari, B., Takacs, Z. K., & Bus, A. G. (2017). What are we downloading for our children? Best-selling children's apps in four European countries. Journal of Early Childhood Literacy. https://doi.org/10.1177/1468798417744057         [ Links ]

Smith, R. (2010). The Long History of Gaming in Military Training. Simulation & Gaming, 41(1), 6–19. https://doi.org/10.1177/1046878109334330         [ Links ]

Stetina, B. U., Kothgassner, O. D., Lehenbauer, M., & Kryspin-Exner, I. (2011). Beyond the fascination of onlinegames: Probing addictive behavior and depression in the world of online-gaming. Computers in Human Behavior, 27(1), 473–479. https://doi.org/10.1016/j.chb.2010.09.015         [ Links ]

Turkle, S. (2015). Reclaiming Conversation: The Power of Talk in a Digital Age. New York, New York, USA: Penguin Books.

van Rooij, A. J., Ferguson, C. J., Colder Carras, M., Kardefelt-Winther, D., Shi, J., & Przybylski, A. K. (2018). A weak scientific basis for gaming disorder: Let us err on the side of caution. PsyArXiv Preprints, 1–28. https://doi.org/10.17605/OSF.IO/KC7R9         [ Links ]

Vygotsky, L. (1930). Imaginação e Criatividade na Infância. Lisboa: Dinalivro.         [ Links ]

Wood, D., Bruner, J. S., & Ross, G. (1976). The role of tutoring in problem-solving. Journal of Child Psychology and Psychiatry, 17(2), 89–100. https://doi.org/10.1111/j.1469-7610.1976.tb00381.x         [ Links ]

Zagalo, N., Carvalho, A. A., & Araújo, I. (2016). Elementos do design de videojogos que fomentam o interesse dos jogadores. Revista ESC - Educação, Sociedade & Culturas, 48, 169–190.         [ Links ]

Zagalo, N. (2017). Narrative design of sadness in heavy rain. Journal of Science and Technology of the Arts, 9(2 Special Issue). https://doi.org/10.7559/citarj.v9i2.246         [ Links ]

Zagalo, N., & Laranjeiro, D. (2018). Brinquedos e jogos digitais para o jardim de infância. In 4.o Encontro sobre Jogos e Mobile Learning. Coimbra: Universidade de Coimbra.         [ Links ]

Zhong, C. B., & Leonardelli, G. J. (2008). Cold and lonely: Does social exclusion literally feel cold? Psychological Science, 19(9), 838–842. https://doi.org/10.1111/j.1467-9280.2008.02165.x         [ Links ]

 

 

Submitted: 5th October 2018

Accepted: 17th January 2019

 

 

Note

[1] The WHO put digital games in its list of health problems, listing them as promoters of "Gaming Disorder". URL: http://www.who.int/features/qa/gaming-disorder/en/

[2] Information on the steps taken by Facebook to create the tool can be found in the release note "Hard Questions: So Your Kids Are Online, But Will They Be Alright?" from December 4, 2017. URL: https://newsroom.fb.com/news/2017/12/hardquestions-kids-online/

[3] Such as the Yale Center for Emotional Intelligence, Connect Safely, Center on Media and Child Health, Sesame Workshop. 

[4] URLs: http://money.cnn.com/2018/01/30/technology/facebook-messenger-kids-letter/index.html or https://www.wired.com/story/health-experts-ask-facebook-to-shut-down-messenger-kids/

[5] https://www.nytimes.com/2018/01/30/technology/messenger-kids-facebook-letter.html

[6] To see all the conferences held since 1996, go to URL: https://icmi.acm.org/2018/index.php?id=related

[7] Another example of the way interaction plays with meaning in games can be found in the TED Talk, "Gaming for understanding" (2011) by the game designer Brenda Romero.

[8] Finishing games is in practice something hard, and a well-known problem in the industry. Every year the issue is discussed in regular media. Examples: "Why most people don't finish video games" CNN, 2011, 

[9] All the studies and publications have been gathered within a unique website fully dedicated to the theory. URL: http://selfdeterminationtheory.org/publications/

[10] "Learn 21: Game-based m-learning for children in kindergarten", 2017-2018, (CENTRO-01-0247-FEDER-009828) 

[11] https://tocaboca.com

[12] https://drpanda.com

[13] https://sagomini.com/world 

[14] https://www.sesamestreet.org/apps 

[15] https://www.edokiacademy.com/en 

[16] http://www.lumikids.com 

[17] https://www.lego.com/en-us/games/apps 

[18] http://www.duckduckmoose.com/educational-iphone-itouch-apps-for-kids 

[19] https://www.minilabstudios.com/apps/arties-world 

[20] https://itunes.apple.com/us/app/leos-pad-enrichment-program-for-preschoolers/id566773525?mt=8

[21] http://gomarcopolo.com

[21] http://monkeypreschool.com

[21]For more about the age groups being developed by the Minecraft education communities look at: https://education.minecraft.net/classresources/lessons/

[22] Newly announced by Time magazine in 2016. URL: http://time.com/4354135/minecraft-bestelling/

[23] Some people became internet celebrities simply by sharing the way they play videogames. These are mostly adolescents, but a large proportion of their audience are children. For more about such popularity see https://www.businessinsider.com/most-popular-gaming-youtube-stars-ranked-2018-2

[25] See more information at the URL: https://education.minecraft.net/