Interaction via emotion

Foto: "Lucid Dream", Elena Kunau and Mariya Yordanova

06.09.2021
Interaction via Emotion

Brain City Berlin

ARTIFICIAL REALITY – VIRTUAL INTELLIGENCE is the name of an exhibition that can be seen from 8 to 12 September as part of Ars Electronica Garden Berlin. This hybrid event presents projects by students of communication design at the HTW University of Applied Sciences (HTW Berlin) in the Reinbeckhallen directly on the Spree – and worldwide in the virtual space. The curators, Professor Andreas Ingerl and Moritz Schell, tell us more about the project and the questions it raises.

The global pandemic has shifted our reality further into the virtual space. Social distancing, the closure of cultural spaces and educational institutions and working in home offices have increased the need for virtual spaces and meeting places. But what do these worlds look like? Who is allowed to participate in them? And what rules should apply here? These are questions that students of communication design at the HTW Berlin have been grappling with. Some of their work was done as part of their Bachelor’s theses, some as project work during the semester. What all the exhibits shown in this ARTIFICIAL REALITY – VIRTUAL INTELLIGENCE hybrid exhibition have in common is that they were created during the lockdown. And they explore cognitive boundaries – by means of brain computer interfaces, voice assistants or a minimally modified VR environment. The “Future Technologies Lab” also features other works that deal with future technologies and new media.

The exhibition is the core of the Ars Electronica Garden Berlin, which was developed this year by HTW Berlin. As in the previous year, Ars Electronica, a festival for art, technology and society, is being held worldwide as a hybrid event. Other highlights of the Berlin Garden include a virtual rave, to which the Ars Electronica Garden invites visitors on 10 September from 9 p.m. to 2 a.m., and the premiere of “Cinemiracle” short films on 12 September, which deal episodically with the themes of the future and society.

Brain City Berlin spoke with the curators, Professor Andreas Ingerl and Moritz Schell (M.A.) from the “Design and Culture” department at HTW Berlin, about the exhibition and the significance of the pandemic for the creative process.

The curators of Ars Electronica Garden Berlin: Prof. Andreas Ingerl and Moritz Schell M. A. of HTW Berlin.

Mr Schell, what’s special about the exhibition ARTIFICIAL REALITY – VIRTUAL INTELLIGENCE that you’ve curated together with Professor Andreas Ingerl for Ars Electronica 2021?

Moritz Schell: The special thing about the exhibition is that the student exhibits on display were all created during the pandemic. They deal with digitalisation on various levels, which has determined our lives much more since the pandemic than before. And it has also made it much more hybrid.

Do you have an example of an exhibit with a hybrid character?

Moritz Schell: In our “Students’ Future Technologies Lab” in the exhibition in the Reinbeckhallen, a robot reacts emotionally to input from humans. However, the input doesn’t work on site, but via a web platform. This means that people can interact with this robot anywhere in the world. On site, we then see the robot’s reaction live. Our rave, which will take place as a highlight of the Ars Electronica Garden Berlin on Friday, is also a hybrid event. Clubbing enthusiasts worldwide can rave along as avatars.

In the student projects, “emotional states become the interface within the virtual world”. Can you explain that please?

Andreas Ingerl: Let’s take the installation “Lucid Dream” by Elena Kunau and Mariya Yordanova. Here, visitors interact with a virtual dream sequence by means of a brain computer interface. The exciting but also strange thing about the project is: you don’t have a controller in your hand. Instead, you wear a headset equipped with sensors. Control works via your mental state. Depending on whether I’m relaxed or tense, I trigger the next dream sequence or not.

Are there electrodes integrated into the headset?

Andreas Ingerl: Precisely. The headset measures brain activity via three contact points. “Emotional” states are then derived on the basis of the data obtained. The headset we use in the exhibition – and partly also in research and teaching at HTW Berlin – is a reduced version of a medical headset. It is already used in the home consumer sector, for example when meditating. Emotional states can be derived using it: relaxation, excitement, concentration etc.

Maria Bürger’s “paradigm shift” object, on the other hand, uses a voice assistant, doesn’t it?

Moritz Schell: Exactly, here the interaction is different. In this installation, visitors speak to an artificial intelligence via a voice interface. A non-linear script runs in the background. And, depending on how I talk to this AI in virtual reality, it tells me a different story. The voice assistant gradually begins to manipulate visitors. The question here is: to what extent will machines influence us in the future – or are they already doing so today?

Andreas Ingerl: As communication designers, we aren’t the ones who feed a large computer with data and then analyse the results. Rather, we’re aware of what an artificial intelligence can deliver. And we play with the effects it has on us – both positive and negative. Ultimately, what is manipulation? Does it lie in my own media competence or in what my opposite number delivers to me? In the case of the “paradigm shift”, the AI basically only reacts to what I provide it with. That means that everything it does is basically intended by me.

What can visitors take away from this exhibition?

Andreas Ingerl: All the installations we show deal with future technologies. However, they address issues of social coexistence and individual influence in the future rather than the purely technical level. They also provide an opportunity to seek direct dialogue with the students – and to continue this dialogue in the exhibition itself. Our aim is to bring the discussion of our questions to the outside world.

You mentioned at the beginning that all the exhibits were created during the pandemic. To what extent does the exhibition reflect that?

Andreas Ingerl: First of all, I’d like to take a little tour: first of all, the Ars Electronica Garden is a very traditional exhibition. You can interact with the exhibits. They are all student works that were created in a time that was defined by restrictions for all of us. Together with our students, we tried to draw positive conclusions from it and move things forward. On the other hand, we were naturally forced by the pandemic to go digital. We had to make the exhibition hybrid-accessible – which is still quite undefined. We, just like the other exhibitors at the Ars Electronica Gardens, experimented and looked for solutions. Because the pandemic was for all of us both a black swan event that came as a big surprise, and a wild card. That is to say, the pandemic actually offered us, both positively and negatively, a huge opportunity to open up new themes and develop them further. It suddenly opened doors to virtuality. Now we have to see what makes sense and what doesn’t. These are exciting fields that need to be examined.

Are follow-up projects already planned?

Moritz Schell: Following ARTIFICIAL REALITY – VIRTUAL INTELLIGENCE, an exhibition is planned in the Futurium. Some of the works shown here will be on display there. That’s why the largest part of this exhibition is also titled Laboratory, because we don’t see it as a completed project. As a predominantly design-based degree programme, we see our work as an iterative process.

Andreas Ingerl: We want to enable students to think not only in terms of art. Or purely commission-oriented, as in communication design. We play in the in-between area. We want to expose students to intellectual and experiential situations that contain both an artistic and applied component. That resonates in all of the exhibits. Basically, we work according to the traditional scientific criteria of quality, but we’re also always looking for the application or provocation.

Do you also work together with other research institutions at the department?

Andreas Ingerl: In Berlin, you’re networked per se after all. Of course there are links, for example to the Free University of Berlin. We want to build that up bit by bit. Our feelers are going out externally. In the long term, this will also be the case with the Art Electronica Garden Berlin. I’ve already brought this topic into our new AI cluster. Discussions are underway about how we can make a contribution to Ars Electronica in 2022 together with the School of Engineering at HTW Berlin. This will then quickly go beyond the boundaries of the university. Ars Electronica has always had a tradition of mixing various disciplines. It’s about scientific findings, but also about artistic explorations of them.

And about creative exchange on an international level?

Andreas Ingerl: Precisely. As part of our preparations, we recently had a virtual meeting with all Gardens participants and presented our concepts to one another. That’s one of the many exciting aspects of the Ars Electronica Gardens: that exchange takes place and new ideas emerge via the platform. (vdo)

Additional informationen and tickets

Ars Electronica Garden Berlin
8 to 12 September 2021, Stiftung Reinbeckhallen, Reinbeckstr. 11, 12459 Berlin.
The online-ticket for the Ars Electronica 2021 costs 9 euros for 5 days.

Ars Electronica Garden Berlin Rave, 10 September 2021, 9 p.m. to 2 a.m.
Free tickets on Rave.Space