Audio-reactive Latent Interpolations with StyleGAN

By Hans Brouwer. Presented at the NeurIPS 2020 Workshop on Machine Learning for Creativity and Design.

This work explores chromagram-weighted latent sequences, onset envelopes, noise reactions, network bending, model rewriting, and musical structure as controls for audio-reactive StyleGAN videos.

• Read the paper (PDF)
• Read the accompanying article
• Watch the supplementary examples
• Source code

The supplementary examples compare earlier audio-reactive approaches and demonstrate the techniques described in the paper.