Product
Kirlian
Kirlian explores how AI can interpret music and turn that interpretation into a visual experience.

The idea
Kirlian began with my curiosity about constructing systems that could use measurable inputs in unconventional formats, music represented through mathematical analysis, for instance, to surface properties that might otherwise remain obscure or difficult to express in other mediums. I wanted to see what a model could make of that material, how it might interpret it, make aesthetic decisions, and carry them into something another person can see, hear and feel.
Music already carries several kinds of meaning at once, from the rhythm, instrumentation and development of the recording itself, to the artist, the circumstances of its making, its references and associations. Then there is everything a listener brings, because the same recording can belong to very different places and periods in different people’s lives.
With Kirlian, I bring these different kinds of information into the material supplied to the model, combining measurements of the recording and its changes over time with research about the song and, optionally, personal annotations. A memory attached to a song is information too, although it is information of a very different kind. What the model receives is a textual account of all this, rather than the audio itself, and that becomes the material it has to interpret.
From interpretation to construction
I approach the translation through two stages, interpretation and construction, beginning with a creative brief in which the model proposes a visual world, its forms, materials, relationships, and how it might develop through the song. Even an increase in energy leaves a great deal undecided, since it could become a denser composition, an unstable surface, or the disappearance of something that had been holding the picture together. What interests me is the choice the model makes, and how it connects that choice to the music.
The second stage takes that brief into code, using Kirlian’s visual capabilities to arrange the elements, define their behaviour, and connect them to the recording. The model can inspect rendered passages and revise its work within the limits of the construction process, with the brief remaining a reference for what it was trying to express. Keeping the stages separate gives me somewhere to look when an idea seems to have disappeared along the way, because I can return to what the interpretation proposed, then examine what was actually made from it.
Once a construction has been saved, the browser plays it alongside the recording, using the song’s timing and audio data. The model has authored the system that produces the frames, so pressing play lets you experience that construction again, without generating a new interpretation of the song.
One recording, a visual world
In the interpretation of Chopin’s Berceuse shown above, the repeating bass becomes the rocking of the whole view, while the melody becomes a strand of silk between a lamp and the moon. As the piece develops, the strand doubles, breaks into beads, grows lace and disperses into luminous dust, before gathering into a single line again.
At 2:23, the lace reaches its fullest development and the view is at its closest, although the interpretation asks for that fullness to remain soft, expressed through finer ornament rather than more glare. By 2:49, the lace is breaking into points, and at 3:50 the strand is whole again, following a sequence whose visual states and timings were authored for this recording.
This is the kind of development I want to explore, because a visual can respond continuously and still feel unchanged, its brightness, scale and movement varying while the relationship between its elements stays the same. Here, one material changes state and remains recognisable through those changes, and returning to a line means something different after watching it come apart.
Building Kirlian around code also makes it possible to work on a particular part of the picture quite quickly, changing a material, adjusting a movement, or fixing something that isn’t working, while keeping the rest of the interpretation. I can make the correction, replay the same passage, and see what it actually did, which makes revision a much more immediate part of working with the result.
The model can work this way too, inspecting a render and revising the code behind it, as it did in this version of Berceuse when it dimmed the moon and deepened the veil’s folds after seeing its first render. Those changes didn’t require a new interpretation of the song, and the two frames below show the revision at the same moment in the recording.
What I build, and what I judge
I designed and built the system around this process, including the way the models receive the music, the tools they can use to construct a picture, and the evidence available to them when they review it. The models author the individual interpretations and constructions, while I shape the conditions, compare versions, review the results, and decide what is worth keeping.
Part of building Kirlian has been learning where to give the model room, where to be precise, and where an apparent creative limitation was actually a limitation I had built into its environment. What I choose to give it matters, whether that’s a personal annotation contributing meaning, a visual direction suggesting how to express it, or a capability that makes an idea possible to construct.
The model’s review records what the rendered evidence supports, contradicts, or leaves unverified, which helps me examine the construction without settling whether it is a good interpretation of the music. A system can execute correctly and still produce something that barely registers on screen, or an elaborate picture that tells me very little about the song, so I still have to return to it as a listener and decide what I find compelling.
Working on Kirlian brings together things that don’t usually come in pairs, non-determinism and aesthetics, creativity and data engineering, etc., and the technical decisions keep bringing me back to what I want someone to experience. I want a visual interpretation to bring something into focus, a relationship in the music, an association I had not articulated, or a familiar song that becomes unfamiliar enough to hear again.
The Kirlian preview follows Berceuse through selected moments, and the demo console lets you explore other interpretations. This is still an experiment in what people and models can make together through music and code, and I’m interested in what someone else might find in a song they thought they already knew.