Self-supervised Audio-reactive Music Video Synthesis

By Hans Brouwer, Delft University of Technology. Supervised by Lydia Chen and Cynthia Liem.

This thesis investigates how to generate audio-reactive music videos without manually configuring the relationship between musical features and a generative model. It introduces an audiovisual correlation metric and uses it to train synthesizers from audio examples, or to optimize an individual video directly.

• Read the thesis (PDF)
• Watch the supplementary comparisons
• Source code
• TU Delft repository record

The supplementary page preserves the original explanation and comparisons between the proposed synthesizers and prior work, including examples for Dwelling in the Kelp, Tau Ceti Alpha, and Temper.