Using game theory to support the audience participation in digital performances

Public participation in art performances has been leveraged by the use of technology as a mediating element between the public and artists. However, technologicalmediation alonemay not be enough to ensure effective auditory participation. In this article, we present the performance “ O Chaos das 5 ”, an audiovisual digital performance with a strong technological support that had the support of game theory to achieve a more effective participation of the audience in their presentations.


Introduction
Throughout the history of Western society, Fine Arts have been characterized by a well-defined role structure, with the artist at the center of the work, while the spectator must passively observe and appreciate, without any direct interaction with the piece. In the case of classical music of European tradition, the division takes place in the figure of the composer, responsible for the creation of the music; the interpreter, a virtuoso instrumentalist capable of performing it with precision; and the audience, who should just enjoy the show and applaud at the end. However, different approaches and techniques were being created over time to break up these roles, engaging the audience in artistic performances.
In the 20th century, some artistic movements began to place the public as part of the show, breaking with the previously established musical tradition. This concept was explored by artists like Allan Kaprow in their Happenings with the group Fluxus (Taylor 2017). The group Fluxus and other artists of the 1950s and 1960s established a structure about an artistic event that can help to create artworks, strongly marked by the participation of the audience. At these events, the audience had tasks to be performed and needed to take part in the show actively.
The Happenings soon left this format of pre-established tasks, and the performance art emerged, where the public takes part in the show on a freeway and not in a guided way as before. An example of a performative concert is 4'33 by John Cage , first presented in 1952, where the public was responsible for the sound result of the piece (Rocha 2005). In this performance, the musician on stage has no control over the sonic outcome as he remains silent. The discomfort generated by this unusual scene is the objective of this work. Thus, performance does not require mastery of the instrument and does not involve sounds generated by it (Santos 2008).
Since the 1990s, artists have been using technology to allow audience participation in the art, using, among other possibilities, a computer network as a tool to connect audience members and artists (Rauen 2011). A recent and promising change occurs in the emergence of the so-called digital performances, which are usually art shows made with the strong support of technology, such as sensors, mobile devices, among others. Thus, . "Using game theory to support the audience participation in digital performances" Per Musi no. 40, Computing Performed Music: 1-13. e204020. DOI 10.35699/2317-6377.2020 in digital performances, mobile devices can be used as a mediating interface in musical performances and compositions, for example, integrating the audience members in the creation and execution of a musical piece.
However, including technologies in artwork could not grant the audience participation in the artistic performance, and some strategies can be used to try to enlarge it. In this paper, we use some concepts from game theory to increase this level of participation. A game is a set of human activities and desires, according to Roger Caillois (1990). Johan Huizinga (2010), who was a pioneer in the sociological theory of the games, presents a comprehensive definition for this field. For him, playing is a reality in which our entire culture originates, defining it as a free, "not serious" activity that takes place outside of everyday life and within its own spatial and temporal limits. Other relevant aspects are that the game promotes the formation of social groups and is not related to material interest.
This current paper expands our previous work, titled "A technical approach of the audience participation in the performance "O Chaos das 5"" Araújo et al. (2019), presented in the 17th Brazilian Symposium on Computer Music, realized in 2019 in São João del-Rei, Minas Gerais -Brazil. Starting with this preliminary concepts about games and artistic performances, and expanding our previous work, we added in this paper three other discussions: • How Roger Caillois's game theory can be seen in an interactive performance?
• How the relationship between games and music interfered in the performance "Chaos das 5"?
• A method to an in-depth assessment of public participation in artistic performances.

The performance "O Chaos das 5"
"O Chaos das 5" is a digital performance created by the ALICE and Ecolab research groups from the Federal University of São João del-Rei (UFSJ). This performance involves the participation of the auditory in its execution mediated by the mobile devices of the audience members. Different from a Happening, in this work there are no instructions for the audience members to guide their interaction during the show. This interaction can vary according to the characteristics of each spectator, composer, musician, and other artists involved. Santos (2008).
Our performance was created using a pre-start moment and three different stages/parts. This pre-start moment counts on the projection of a stopwatch informing the time remaining to the beginning of the performance ( Figure 1). In this projection, there are also some instructions about how to connect to a local wifi network and access a website. On this website, there is more information about the event. Meanwhile, some performers from our group interact with the audience members, showing how to use the computational system that was developed for this very event.
When the countdown finishes, we started the first stage, which lasts about 9 minutes and consists of the creation/demonstration of synthetic images and sounds. In the second stage, which lasts 10 minutes, the ambient sound starts to simulate immersion in the audiovisual landscape of daily life in large cities, highlighting concrete sounds, forming a dense, irregular, and aggressive soundscape. Finally, the third stage lasts 10 minutes and is based on an atmosphere of calm and surreality. . "Using game theory to support the audience participation in digital performances" Per Musi no. 40, Computing Performed Music: 1-13. e204020. DOI 10.35699/2317-6377.2020

The Three Layer's of the Performance
Aiming to provide an immersive experience for the audience, it is possible to report the performance throw three layers of information: a Gestural Layer; a Musical Layer; and a Visual Layer. The combination of gestures and improvisations from 5 artists physically interacting with the audience characterizes the Gestural Layer and defines the physical space of the event. Initially, the performers gather among the audience, making it difficult to distinguish who is part of the audience and those that are the artists involved in the scene. This distinction becomes clearer during the performance, especially at a time when the performers take off part of their costumes and invite the audience to interact with them, painting their bodies with pens and ink ( Figure  2). Our second layer is the Visual Layer, which is composed of projections and applications developed on demand for this very production. These applications create digital images in real-time and present an aesthetic that puts the computer on the scene, opening the "Black Box" and exposing the machinery in its imagery making. The Visual Layer refers to the paradigm "painting through the action", according to Ribeiro (2010), . "Using game theory to support the audience participation in digital performances" Per Musi no. 40, Computing Performed Music: 1-13. e204020. DOI 10.35699/2317-6377.2020 and in this performance, the painting is digital. The actions taken to create the projected images are presented to the public, and the software code is designed together with its visual result, as in a performance of live coding (Collins et al. 2003). Besides that, two programmers are in charge of modifying the code and visual projections (Figure 3). Webcams and image processing in real-time, which capture the artists and the audience, complete the visual configuration of the "O Chaos das 5". Finally, in the Musical Layer we have two sorts of participants: musicians that are members of our group and audience members that want to play along with us. The musicians, who are located around the space, are responsible for creating the sound base of the spectacle. They played several electronic unusual instruments, created for this performance, using as interfaces mouses, keyboards, joysticks, MIDI controllers, as well as electric guitars. The audience members can play using Digital Musical instruments (DMI) on their mobile devices, available on our website ( Figure 4).
The Musical layer, in the first stage, is composed of synthetic sounds, with a predominance of musical material formed by glissandos, which were inspired by Metastasis of Iannis Xenakis, representing the vertigo of a free fall. In the second stage, there is a sound dialogue in 5 layers: the sound of the urban scene projected on a video; an improvisation with samples fired and manipulated on an instrument created for the performance; 2 electric guitars making specific and improvised interventions; samples played from the public's mobile devices, representing sounds of sirens, bells, automobiles, among others (third instrument in Figure 4). The last stage is based on a mix of synthetic sounds and real instruments, using known sounds to bring all the participants to a calm atmosphere, based on a distorted quote from the Quatuor pour la fin du temps movement by Olivier Messiaen. . "Using game theory to support the audience participation in digital performances" Per Musi no. 40, Computing Performed Music: 1-13. e204020. DOI 10.35699/2317-6377.2020.32675

The technology behind the performance
In pre-start part, the users/participants name is requested when they access the website. These names are used in our show at the end of the performance, when a projection with the credits, like those used in Movies, shows the name of all participants of the performance, including the audience members' name. Thus, they are recognized as part of it. After informing their names, users wait on a static page that has a counter synchronized with the projected stopwatch. When the counter/timer resets, all client-side applications send a page redirection request, releasing the DMIs to the public. As soon as the instruments are made available, the first stage of the performance begins.
Using the DMIs provided on the website, the audience members can contribute to the musical layer of the performance, firstly playing different types of glissandos (first instrument in Figure 4). These glissandos can be configured by the users using a few parameters provided by the GUI. Another instrument can be played using the accelerometer of mobile devices as a control to synthesize some chromatic melodies (second instrument in Figure 4). The third available instrument (third instrument in Figure 4) is based on samples and can be played using a set of buttons in the mobile devices' GUI.

Discussion: Games and musical performances
Despite provoking a positive change, audience participation in the theater brings with it some problems, as discussed in "The emancipated spectator", by Rancière and Elliott (2009). Concerning the separation between stage and audience, the author considers that viewers are passive, as they are devoid of the possibility of acting. He also considers that "looking" is a passive act that is the opposite of "knowing". In contrast, the action brings the viewer closer to knowledge. Therefore, he demands a new theater, without the spectator's condition, in which active participants engage in collective action. In this case, the show takes place through the participants' agency. An interesting approach that can help in this engagement is to use games to structure public participation in interactive performances ).
That said, this comparison is feasible because in some cases the field of music can be seen side by side with the field of games, from popular ones that are based on children's folklore to the electronic ones, where music Araújo, João Teixeira; Silva Júnior, Igino de Oliveira; Costa, Rômulo Augusto Vieira; Almeida, Marcela Alves de; Schiavoni, Flávio Luiz. 2020. "Using game theory to support the audience participation in digital performances" Per Musi no. 40, Computing Performed Music: 1-13. e204020. DOI 10.35699/2317DOI 10.35699/ -6377.2020 helps in the composition of the scenario and the immersion of the players. Zehnder and Lipscomb (2006) also state that music can suggest changes in the narrative or plot of a game, act as an emotional signifier, improve the sense of aesthetic continuity and cultivate thematic unity.
It is also possible to highlight points in common between the two practices, games and music, such as cognitive responses to certain events and the presence of the virtuous player. In music, virtuosity refers to that musician with complete mastery of musical theory and practice applied to a musical instrument. In games, virtuosity could be defined as a player with human capabilities that are beyond normal people and that uses specific techniques and gestures. The virtuosity also has four characteristics that can be very useful for musical presentations and other artistic performances, such as a) urgent optimism, which is the hope of success when faced with an obstacle; b) social connections, created by participating in a group activity, thus strengthening the bonds of trust and cooperation; c) pleasant productivity, where those involved are likely to work hard to obtain satisfactory results; and finally, d) the epic meaning, which is the desire to be involved in something big and with a definite objective McGonigal (2010).
Dealing specifically with the relationship between the game and the musical performance, it is clear that both occur outside of everyday life, in addition to being free activities that reinforce the playful aspect. Players and musicians often organize events in time and anticipate moves to ensure that the outcome occurs at the expected moment. Besides, in both cases, whoever is performing, the activity is guided, in most cases, by auditory stimuli. To organize these tasks precisely, practice is necessary, bringing up the figure of a virtuous player.
But the similarities do not end there. The two practices do not intend to generate profits, do not present a previous result of the process, and respect rules established within the defined time and space. Beyond it, game and performance, while artificial realities, establish rules for human interaction. In a performance that brings together audiences and artists, the rules of interaction guide the actions of the participants, becoming a fundamental element in defining performance. However, it is worth noting that simple and prescriptive rules can generate public disinterest, while very complex rules are difficult to understand and, therefore, can become a problem for engagement.
A relevant aspect being analyzed is to assess the level of rationality of the participants. It is observed that lower levels of rationalization favor public interaction, participation, and engagement. In Caillois's theory, mentioned earlier, rationality oscillates between Paidia -improvisation related to play, and Ludus -related to formal games. The spirit of Paidia and Ludus fluctuates according to the level of rationalization. When more structured and pre-defined, more Ludus. The more indeterminate, the more Paidia is the game (Vieira et al. 2020).
To further exemplify this relationship, the authors used a subdivision of games, also from the perspective of Caillois. They are: âgon -competition, refereed to sports games, where dexterity in the execution of a certain activity prevails; alea -defines games related to luck; mimicry -simulacrum, indicating games that consist of imitations; and ilinx -vertigo, a category which depicts games that affect the body.

Applying games theory in our performance
When we defined our performance, we also defined some goals to try to reach when staging it, especially concerning the audience's participation in it, like: Araújo, João Teixeira; Silva Júnior, Igino de Oliveira; Costa, Rômulo Augusto Vieira; Almeida, Marcela Alves de; Schiavoni, Flávio Luiz. 2020. "Using game theory to support the audience participation in digital performances" Per Musi no. 40, Computing Performed Music: 1-13. e204020. DOI 10.35699/2317-6377.2020.32675 • we did not want to define a stage or a physical place where the audience would be and where the performance would happen; • we did not want to teach users how to use the provided instruments; • we wanted to have people taking part in the piece, interacting with our gestural layer; • we wanted that every audience member could have an individual and unique experience taking part in our performance.
Thinking about our first goal, we would like to have people in every place and to perform among the audience members. All we had defined were where to place musicians, projectors, and live coders with laptops to the projections. However, when the performance began, and we turned the projectors on, this arrangement ended up defining the audience space. People should walk around and through the performance during the presentation, and this goal was not easy to be reached because audience members used to avoid being in front of the lights of the projectors and used to hide in the shades of the space.
We started to use some concepts of games to achieve this goal like using characteristics of free games and keeping a playful mood. At the beginning of our show, when accessing our website, the participants got into a kind of game in which they need to recover 3 passwords that are exposed all around the space. This interaction at the beginning of the act is physical/digital since the public is "forced" to move around the environment, looking for hidden passwords for the mobile devices. The search for these passwords generated competition among some of them, allowing us to think of Agon as an aspect that motivated participation. As it is the category that deals with the sportsmen, it is also possible to compare these with the virtuous musician, where both achieve technical excellence through training and repetition.
Our second goal presented a challenge to achieve because, according to Hindle (2013), it is highly recommended to provide information about how to interact with musical interfaces, especially if the audience is diverse and heterogeneous. Since we could not preview who would be our public, we could hope that a certain part of it might have difficulties and issues when dealing with technology. However, the instruments were implemented focusing on laypeople in music, based on simplicity and usability. Thus it was not necessary to have exhaustive technical explanations on how to play the instruments (Bin et al. 2016;Lee and Freeman 2013).
Beyond, to reach the second goal, another concept of game theory was used, called Mimicry, by Caillois (1990). Mimicry games have more flexible rules and are more open to creation since the sequence of gestures is not prescribed and depends on collective interaction. Members of our group were around the scene space, acting as members of the audience and helping people that could need some technical support, doing the interaction approaches of the make-believe games. Instead of teaching people how to use the application, they were using the mimicry concept, using the application and convincing people to do the same.
This concept was also used by the gestural layer of the performance when the performers used to first paint their bodies and the bodies of other performers, and then ask the audience members to do the same. When we took the first step in this task, we broke the shyness of the audience members, that felt more comfortable just mimicking the gesture of the performers, taking a more active part in the presentation.
The result of the soundscape created in the musical layer resembles the alea type of games. Alea is the game of chance in which the result is random and depends on some luck choice. Therefore, the final result presents . "Using game theory to support the audience participation in digital performances" Per Musi no. 40, Computing Performed Music: 1-13. e204020. DOI 10.35699/2317-6377.2020 a certain level of irresolution, which depends on the way each user interacts individually with the interface, reminder, for instance, the 4'33 piece from John Cage. Thus, audience members could change the sounds that they were playing, changing the collective soundscape outcome. Despite this, it was not a random process and people could decide what they would like to play, the final result can be considered very aleatory. The final sonic experience depends on where a person is located in the scene space and the result would be an individual experience for each person.

Evaluation of public participation
Once we could increase the participation of the audience in the performance using game theory, it was necessary to find a form to evaluate this participation. Mazzanti et al. (2014) cite some metrics to evaluate platforms aimed at participatory performances: • Control Design Freedom: which describes how free public interaction can be provided on the platform; • System Versatility: referring to an overview of how simple is the performance and comfort of the artists on stage; • Audience Interaction Transparency: regarding the clarity of the relationship between the manipulation of the audience and its effects; • Audience Interaction Distribution: which is how much interaction can be located concerning the participants (strongly centralized interface vs. each participant using an interface); • Focus: regarding how easily the audience can freely focus on different aspects of the performance (the stage, its interaction, visuals, music, etc.); • Active/Passive Audience Affinity: referring to the difference between active and passive participants.
Following these metrics, we can assess that public participation in our performance has a high degree of Control Design Freedom since the instruments could be accessed at any time, at any stage. Also, since we assumed a playful mood and the characteristics of the free game, people could occupy any space in the performance scene and interact with every performer during the presentation. However, in our first performance, this degree of freedom caused some discomfort due to the use of instruments at unexpected moments, generating undesirable noises during the show. Assuming the Mimicry game idea, members of our group started playing the "correct" instrument in each part of the performance, showing to the audience's members how to play it and how to interact with it. Still thinking on the Control Design Freedom, it is possible to assume that the lower the predetermination in the control, the closer the interaction is to the Paidia type games, and the more predefined the interaction control is, the closer to the Ludus type games. We realized that the interaction mode is directly linked to performance objectives. In this way, using various strategies to fulfill different objectives gives the audience more options to interact and the way they feel most comfortable.
The System Versatility was given as expected, but we identified some issues concerning the network capacity, more precisely, the scalability of the network. Several related works use the public participation based on smartphones, characterizing it as a DMI, as is the case of works * 12 *, by Egozy and Lee (2018), SWARMED, by Hindle (2013) and CROWD IN C[LOUD], by de Carvalho Junior et al. (2016). The applications presented in these works are server-centered, which can cause problems regarding scalability. This problem happens if the system is accessed by several users that go beyond a supported threshold. This aspect has impaired the versatility of the system and is still a challenge to be overcome. In much of the related work, the number of participants in the public was limited to avoid problems of this type, a solution that we discarded since we did not intend to push this kind of limitation. Again, the playful mood helped to work around this situation, suggesting to people to play together with other friends that were not dealing with network issues, for instance.
The system implemented ended up being quite adequate when analyzing the Audience Interaction Transparency. Since the instruments developed have some configuration parameters, the participants could look for an individual setup and have a very casual relationship between a gesture, the configuration, and the generated sound. Furthermore, we noticed that it was not necessary to teach how to play the instruments. To inform how to connect to the network using the projection was enough to help most of the participants. Our group playing with the instruments and being mimicked by the audience members, without teaching how to play it let the audience members explore the instrument playing it. We also believe that the very easy-to-use interface of our instruments was a very interesting decision taken in the design of the interfaces.
The Audience Interaction Distribution was achieved through the existence of more than one instrument for each moment in the play. People could choose an instrument and there was no restriction to access it anytime. When assuming the possibility of artistic performance also becoming a game, participants can be subject to luck, which through some method, indicates whether they can participate in the play or not, what objects they can use, and what role they should play. This is a characteristic of alea and in the play O Chaos das 5, it was present in the choice of instruments, which suffered interference from the mimicry of the actors. Thus, the "luck" of being inspired by one or the other actor would define what the behavior would be until the end of the play. For those who were not inspired by anyone, there was the fact that the instruments were all available on the Web server.
As for the Focus on performance, this was a very critical point in our system. Part of the audience was distracted, trying to play, and could not pay attention to the performance. This ended up generating a metaphor for the modern life where we fail to see what happens around us to try to perform actions on our smartphones. Once we chose to use the alea type game to engage people to move around, it was not possible to keep the focus on a single part of the performance. Also, the interactions from an artistic presentation can present different levels of Ilinx, varying according to the gestures necessary to produce the expected effect. It is important to note that the focus is not on how these effects are triggered or what experience they provide, but how it happens concerning body movement, more connected to the artists. In the play presented here, Ilinx was present in the vertigo of exploring the place and discovering the passwords that start the play.
Considering the Active/Passive Audience Affinity, it was observed that initially, the public participation was very shy. Again, we used the strategy observed in Mimicry games to remedy this. We infiltrate some actors among the audience and these actors used to take the initiative to interact with our performance. From the imitation of these actors, the audience members began to interact with the presentation, breaking the shyness and without having to go through a tutorial or follow a predefined set of rules.

Final considerations
The performance "O Chaos das 5" was the first attempt to create an audience performance developed by our research group. . "Using game theory to support the audience participation in digital performances" Per Musi no. 40, Computing Performed Music: 1-13. e204020. DOI 10.35699/2317-6377.2020 Our first presentation was in the vernissage of an exhibition in the Art gallery of our University. Thus, the main part of audience was artists, students, and professors of Art, people that know what is a performance and that could probably take part in it. However, the audience's participation was very shy and, since this premiere, we noticed that it would be necessary to adopt some strategy to grant the audience's participation in our performance.
Initially, the possibility to create instruments with the web audio API motivated us to use the audience cellphones as spread loudspeakers, giving the audience members the chance to take part in the soundscape of the performance. However, it was not enough to grant the audience participation, and we started looking for some tools to help us to reach a better result on it. The found solution was to use game theory and the concept of different types of games to try to reach better participation in other presentations. The result, presented in this paper, is a performance with a high playful mood that was experimented a few times, each time with different outcomes and experiences.
The participation of the audience was an interesting social experience and, somehow, at an unexpected level to us. People reported to us that they would like to take pictures of the show, but it was not possible since they were using the cellphone to play the DMI. The experience of having the cellphone "stolen" or "kidnapped" for a while had certainly a social impact on part of our public.
We experimented with some technical issues caused by the number of participants. Certainly, some people could not connect to the instruments, and it resulted in an interesting side effect. At some point in the performance, people did not know if they should try to use their cellphones and take part in the performance, or just watch and enjoy it, paying attention to what was happening around them. It resulted in an interesting metaphor of contemporary life, where the anxiety of being connected all the time sometimes deprives us of observing what is happening around us.
The creation of this performance involved up to 20 people from different areas, knowledge, and skills. Also, people with different levels of experience in live performance, technology, music, and so on. Even so, the differences here were added up to make the performance viable. The integration of such an interdisciplinary and heterogeneous team was an amazing experience for all the participants. During the performance, the computer guys were struggling with the technical questions, improvising, coding and setting up the server, finding fast solutions to network jam problems, understanding and solving computer problems on the fly. There was always an audience member asking for help, trying to do something while the show was happening. At the same time, the performers and the musicians were also improvising, dancing, playing, and acting with the situation, in the middle of the public, keeping it rolling because, in the end, the show must go on and it was a live performance. Maybe only an open live performance with audience participation can allow trying out the improvisation at this level.
The present research is still incipient, and certainly, there are several other ways to explore games theory to support audience participation in artistic performances. We know that our experience is not enough to validate this model, and it will be very interesting to work more on it to have other study cases trying to validate our purpose. As future work, authors intend to apply this model as a teaching methodology, involving children in art creation using games theory and technology.