Visual Lyrics

Turn music lyrics into animated visuals

Demo

Visual Lyrics is an automatic animated lyric video generation pipeline with a text-driven interface for creative control. See how it works.

Semantically-Matching Stylizations

Three distinct ways of stylizing lyrics: image, animation, and visual.

🖼️

Image

Image stylization generates a supporting graphic for the video. They can be used for words that are visually-concrete objects or abstract metaphors that can be associated with concrete objects.

Animation

Animation stylization animates the word itself. They can be applied to words related to motion or words sung with special vocal attributes like upwards or downwards pitch shift, word elongation, and vibrato.

🎨

Visual

Visual stylization modifies font attributes, such as font family, size, color, or rotation. They can be used for words related to size, color, emotional qualities, or depending on the energy of the vocals (loud/quiet).

Example Results

Lyric video results

🔊 Videos include audio

Hip-hop/Trap (multilingual) (ZOOM)
Rap/Rock (Lose Yourself)
Disco/Pop (Espresso)
Indie/Electronica (Fireflies)
Pop/R&B (7 Rings)
Trap/Rap (Money)
Rap/Hip-hop (portrait video, multilingual) (TEAM TOMODACHI)

More examples of creative stylizations

Failure Cases

🔊 Videos include audio

Multiple simulataneous vocal lines (texts overlapping each other)
Very fast rap (animation cut off early)

Dataset

We collected a dataset of 306 code-driven creative text animations to serve as inspiration for our generation pipeline.

Dataset Preview