Drawing with sound

The world is drawn by the sound of your own voice. That’s the line I keep coming back to.

The first version of the shader was a pulse. The bat chirps and a visible ring of light radiates outward from her at the speed of sound, and the world reveals itself as the ring passes over it. Past the ring, darkness. Inside the ring, the world. The wavefront is literally the wavefront. I had a build of this running and in still screenshots it was one of the prettiest things I’ve made in Unreal. Everyone who saw it wanted to play it.

The problem is bats chirp constantly. The problem is sound moves at about 340 m/s, which at game scale crosses the playspace in under a second. The next chirp fires before the last ring has faded, and the one after that fires before that one has faded, and the level becomes a strobing onion of overlapping wavefronts inside which nothing can be read. You can’t see the alien through the rings. You can’t see the cave through the rings. The bat is hunting in her own party trick. Cut.

The second version mapped surfaces to color by their material’s resonant frequency. Stone one hue, wood another, metal another, flesh another, and the chirp lit them up by their tuning. I built it, walked through the level, and it just felt off. I think the honest read is that color is for things that matter, and “what is this surface made of” is not a thing the player needs to know in a stealth game about not getting shot. I let it sit for a few days. Didn’t warm up to it.

Both versions were trying to be photographic. The world looks like a world, then the chirp does something to it. The thing that broke the problem was deciding the world doesn’t look like a world. The world looks like a drawing. Steady, monochromatic, held in place. The chirp doesn’t reveal a hidden image. It draws on the one that’s already there.

Once I let go of the chirp being the light source, the shader collapsed into something I could actually build. The screen is paper. Surfaces are pencil. The chirp is the hand.

What’s in the build now is a sketchbook pass. Cross-hatching for shadow density, contour lines along curvature, a paper-grain noise that breathes a little so a still frame doesn’t look frozen. Everything is one color value, and the color value is set per scene by whatever is emotionally true for that night. Cave nights warm. The military quarter cold. The bat doesn’t choose the color. The night does.

Sound is the only thing that introduces light into the drawing. A chirp brightens what it touches for a held moment, like a lamp passed over the page. A scream from a civilian flares the area around them. Footsteps from an approaching alien write themselves onto the floor near the bat’s perch before the alien is in sight. The player isn’t seeing through the bat’s eyes. The player is seeing the page the bat is hearing.

The thing this fixes, that the previous versions kept failing on, is that information now persists. A chirp leaves a mark for as long as the mark is useful. Two seconds of grace before the drawing settles back. The bat can chirp, decide, and move, instead of chirp and squint.

Next is figuring out what the page does when the bat is bitten, or hungry, or close to dawn. Right now it’s steady. It doesn’t have to stay that way.