The case: ONSTAGE.ai is a company with a mind set on revolutionizing streaming and recording of classical music concerts. The goal is to bring the audience closer to the music, by enabling music halls to produce high-quality streams in an affordable price.

The challenge: In order to create high-quality stream, one that attempts to recreate the excitement and captivating moments that one would experience in a physical concert hall, it should include highly customized video content. This is traditionally achieved by involving video stream director, and multiple camera operators. Such setup results in high costs and low capacity for producing video streams. The goal of the project was to provide automated video content selection, based on what is happening in the music.

The solutionONSTAGE.ai has integrated artificial intelligence algorithms to act as an experienced producer and camera operator. The AI system automatically selects video content (such as focusing on a conductor, a soloist, or a particular instrument player in the orchestra) based on what is happening in the music. This creates a dynamic and engaging experience for the audience while watching a live stream at home or on the go. At the same time, costs of producing such stream has been drastically reduced.

The technology: novel automated score tracking algorithms, machine learning on audio tracks, automated image recognition, camera tracking and movement control.

ConclusionsThe end result is the ONSTAGE.ai application, which acts as an experienced producer and camera operator, and is the first tool on the global market providing AI-supported music following and automated video streaming solution. The ONSTAGE app – AI-supported tool premiered on the 17th of November 2021 at the Lucerne Festival.

Learn more