In the final part of our series on the changes brought about by interactive music, we’ll explore how this groundbreaking format could redefine the way listeners experience music. Throughout history, humanity has found joy in music. The mediums enabling us to repeatedly enjoy live performances have evolved continuously—from vinyl records to cassette tapes, CDs, and today’s online streaming services.
The first online music streaming service, RealAudio, was launched in the United States in 1995. As the first commercial service capable of streaming music and audio even on low-speed internet connections, it marked a significant milestone. Since then, music streaming services have become the primary means of music consumption. However, for quite some time, there has been no significant evolution in the mediums providing new types of music experiences. Enter interactive music—a next-generation medium poised to revolutionize stagnant patterns of music consumption.
Interactive music seamlessly integrates visual elements, storytelling, and interactive motion with sound, offering a multisensory experience. Powered by generative AI, this innovative format connects dynamic and captivating visuals with sound, often within metaverse spaces or apps. How might this groundbreaking experience transform the way listeners engage with music?
1. Music That Evolves at Your Fingertips
Interactive music unfolds alongside visuals and stories, creating a dynamic and immersive experience. Leveraging generative AI technology, sounds adapt to the listener’s movements, with corresponding visuals appearing in real-time. Within metaverse environments built around an artist’s unique vision, evolving scenarios influence the progression of sound, allowing for an ever-changing musical narrative. Listeners can engage with this innovative format through simple, intuitive touches, enjoying music that reacts directly to their input. As avatars move or interact with 3D objects within the metaverse, the flow of music shifts accordingly. Similarly, app-based interactive music enables users to experience dynamic and stunning 3D visuals synchronized with sound, transforming traditional listening into a captivating multisensory journey.
A prime example of this new wave in music is the interactive single ‘FIGHTMAN,’ a collaboration between Verses and SUMIN. ‘FIGHTMAN’ is an innovative app that empowers users to actively create and modify music in real-time, delivering a truly interactive musical experience.
The app’s interface is designed to be intuitive, allowing users to explore various features effortlessly. For instance, dragging a slime shaped object triggers a beat repeater effect, while touching the background adds unique audio effects. These elements blend seamlessly with SUMIN’s music, creating an immersive and engaging experience. Listeners move beyond passive enjoyment, gaining the opportunity to reinterpret and co-create the music, adding their own personal touch.
With ‘FIGHTMAN,’ technology and art merge to introduce a groundbreaking form of musical engagement. This new approach lets users actively participate in generating music, making the experience both creative and rewarding.
2. Crafting Your Own Remix: Easy and Fun
Interactive music allows listeners to create unique remixes that reflect their personal style with simple and intuitive controls. Not only can users remix tracks, but they can also produce short-form music videos that complement their creations. Powered by generative AI, the sounds adapt to the listener's screen touches and avatar interactions, resulting in personalized music remixes. Additionally, tools like Verses’ ‘Beat-based AI Music Video Generator’ enable users to create music videos perfectly matched to their remixed tracks.
All of this is achieved through intuitive interactions—listeners can express their emotions and ideas with just a few touches, without needing complex tools or technical skills. Interactive music sparks creativity and provides an excellent medium for transforming imagination into tangible, immersive experiences.
A great example of this innovation is Verses’ ‘DJ Hitchhiker's Metaverse Music.’ This interactive music platform makes it easy for listeners to craft personalized remixes. Within the app, users take on the role of a spacesuit-clad hitchhiker, exploring various virtual music environments. By touching the screen, they can manipulate beats, add rap elements, and ultimately create their own unique music compositions.
3. Immersing Yourself in a Musician’s World
Interactive music in the metaverse is built around scenarios that reflect a musician's stories and messages. Listeners can step into a 3D world that brings these narratives to life, allowing them to experience and connect with the artist's vision firsthand. Just like playing a game, users can customize their avatars, explore the metaverse, complete missions, participate in events, and purchase virtual items.
By engaging as characters within the musician's universe, listeners can experience the artist’s music and stories simultaneously. Activities such as buying exclusive merchandise, sending messages to the musician, or taking selfies with the artist’s avatar enrich the overall experience. These interactions not only provide immersive content but also allow fans to express their admiration for the musician’s creative world more actively.
One remarkable example is Björk’s interactive art project ‘NotGet VR,’ based on the track “NotGet” from her album Vulnicura. Released in 2017, this virtual reality experience showcases Björk’s avatar transforming into various forms within a dark, mystical environment. As participants enter the VR space, they encounter a fusion of music and visuals, enhanced by motion-capture technology that animates Björk’s avatar in dynamic ways. The music evolves alongside the visuals, creating a deeply engaging and transformative experience. ‘NotGet VR’ enables fans to participate in and connect with Björk’s artistic universe on a profound level.
4. When Play Becomes Creation
Interactive music offers an engaging experience where even the simplest interactions bring music, visuals, motion, and storytelling to life. As listeners manipulate these elements, they transition from passive consumers to active creators, shaping and reimagining the content themselves.
Through intuitive controls that make music and visuals evolve dynamically, interactive music allows listeners to experience the magic of turning play into creation. This accessible yet awe-inspiring medium can inspire listeners to explore music creation further, encouraging them to craft original works that reflect their individuality.
A prime example of this concept is ‘Incredibox,’ an app that provides users with a variety of beatboxer avatars. Each avatar represents unique sound effects, melodies, or choruses, enabling users to create original music by dragging and dropping different combinations. The app’s intuitive interface makes it easy for users of all ages to enjoy music creation, and it is even used as a fun tool to teach children the basics of music composition. ‘Incredibox’ transforms the listening experience into an opportunity for creative participation.
Another notable example is ‘Vidiyo,’ an augmented reality app by LEGO. ‘Vidiyo’ empowers users to produce their own music videos by featuring “Bandmate” minifigures alongside custom sound and visual effects. Users can film their videos, add creative elements, and collaborate with friends to make vibrant, imaginative content. This playful process of creation highlights how interactive music bridges the gap between entertainment and artistry, fostering collaboration and creativity in a fun and engaging way.
Both of these examples illustrate how interactive music opens up new opportunities for listeners to become creators. They demonstrate how play can seamlessly transform into music creation, enriching the overall musical experience.
5. The Ultimate Joy in Music
Interactive music maximizes the joy of music by offering a rich, multisensory experience that blends sight, motion, and storytelling. Listeners are immersed in a synesthetic experience, where music stimulates not only their sense of hearing but also their sight and touch. As sounds visually manifest and evolve with the listener's interactions, the experience becomes even more engaging and immersive.
For music lovers, this is an unparalleled experience, while for those less interested in music, it offers a new, stimulating way to connect with sound. Whether it’s the energy from lively tunes, the comfort from soothing melodies, or the emotional release from lyrical music, interactive music enhances the gifts that music provides. It connects people and expands the possibilities of healing and self-expression through sound.
One example of this is ‘MindTravel,’ a project led by pianist Murray Hidary that combines music with mindfulness to create an immersive experience that offers emotional and multisensory musical enjoyment. Typically held in nature, participants listen to live piano performances through headphones while enjoying the surrounding beauty or engaging in activities like walking, which promotes psychological stability and healing.
The core of ‘MindTravel’ is the "Silent Piano Experience," where listeners are free from external noise and can explore their inner selves through music. In this format, music becomes more than just background noise—it connects with the listener's emotions, guiding them to create their personal musical journey. For avid music lovers, this experience provides deep immersion, while it also offers a new emotional connection for those with little interest in music.
‘MindTravel’ engages all the senses—hearing, sight, and touch—while allowing participants to experience the healing power of music in harmony with nature. Through this synesthetic experience, people not only feel the comfort and solace music provides but also connect with others, expanding the potential for healing and enriched lives.
