A Mirror for Emotion in the Age of AI
Since 2019, Mienophone, a collaboration by artists Zhe Wang and Shang Heitz, has explored the profound connection between music, emotion, and emerging technologies. Music often shapes our feelings, making us feel deeply influenced by its melodies. But what if we could shift this dynamic, creating a feedback loop where our emotions directly inform the music we hear?
As Machine Learning Algorithms advance in their ability to detect human emotions, Mienophone asks: Can this technology act as a mirror, reflecting our inner states through sound? Beyond simple detection, can AI truly "understand" emotion in a way that resonates with human experience? Emotions are fundamental to human social relations – they are what truly differentiate us from artificial intelligence. How does this interplay between human expression and algorithmic interpretation reshape our subjective experience? Could it offer a novel pathway to self-awareness and a deeper understanding of our own affective landscapes? And what does this mean for the future of art and the artist in an increasingly technologically mediated world?
These are the key questions Zhe Wang and Shang Heitz explore through the Mienophone installation, video works, and photographs. The project is supported by the KARIN ABT-STRAUBINGER Stiftung, with technical support from Berlin Glassworks.
---
How the Mienophone Installation Works
The Mienophone installation features a unique glass sculpture and a custom-built software synthesizer. A camera captures the viewer's face, and an emotions API running locally analyzes the image to detect emotions. This analysis is based on Paul Ekman's groundbreaking research from the 1960s and 1970s, which identified six basic, universally recognized emotions: happiness, sadness, anger, fear, surprise, and disgust. Ekman believed these emotions are innate and vital for human survival and social interaction. Each detected emotion influences the synthesizer's sound, letting viewers hear their own expressions translated into music.
The camera sits inside an organically shaped glass sculpture that intentionally distorts the viewer's image. This distortion makes us wonder if AI truly "sees" us clearly, and, more fundamentally, if it can "understand" emotion in a way comparable to human empathy and social connection. The sculpture's human-like form also mirrors how we often imagine AI – a fragile, infant-sized device that makes us feel like parents to this new intelligence, grappling with its potential for understanding.
Glassmaking is ancient, while computer chips are new, yet both have had an immense impact on human society. The rapid progress of Machine Learning Algorithms is deeply tied to humanity's future. Will we achieve a peaceful coexistence with superintelligence, or will an all-powerful AI control a society where human creativity is lost?
---
Emotions Explored in Video Works
"once I showed you what I want"
In "once I showed you what I want," Zhé carefully dissects performances, images, and emotions from old movie close-ups. Emotions are often expressed instantly through complex facial movements. The work also explores how human emotions can be incredibly complex, like self-deprecating laughter mixed with deep sadness. Can AI truly grasp these subtle nuances, or does it just give us simple percentages like "99.9% happy"? This piece challenges the very notion of what it means for AI to "read" or "understand" something as profoundly human as emotion and its role in social interaction.
"let's laugh together"
"let's laugh together" focuses on laughter, drawing inspiration from Theodor W. Adorno's insights in Dialectic of Enlightenment:
"Conciliatory laughter is heard as the echo of an escape from power; the wrong kind overcomes fear by capitulating to the forces which are to be feared. It is the echo of power as something inescapable."
Adorno highlights laughter's role in our interactions and power dynamics. We laugh at our weaknesses, our ignorance, and in response to others' laughter, revealing complex social connections. This piece further probes whether an algorithm can truly process the social and psychological layers of such a fundamental human expression, a domain deeply rooted in our shared humanity.
---
Mienophone invites viewers to deeply consider their own emotions, the pervasive influence of technology, and the evolving nature of art in the age of AI. What might we discover about ourselves when our emotions are not just observed, but actively reflected and interpreted by an artificial intelligence, prompting us to question the very nature of its "understanding" and the unique place of human social emotion?