Paint That Squishes

How the paint in Fold & Squish splats, mixes, squishes and sparkles, and how Metal keeps it all running every frame.

In Fold & Squish you paint half a butterfly, fold the page shut, give it a rub and peel it open. All the paint is simulated on the GPU using Metal, and this post walks through how that works. It's mostly pictures.

In INKS (2016, Apple Design Award) a pinball left a trail of splats behind it. Fold & Squish comes from the same place, but this time the paint is thick and has some weight to it.

INKS pushed the phones of 2016 about as far as they would go. Ten years on, Apple's GPUs, Metal compute and unified memory leave room for a lot of things I wanted back then and couldn't fit in.

Each sheet is a stack of 1024×858 textures. There's wet paint that can still move, dry paint that has soaked into the paper, glitter, and the texture of the paper itself.

The painting. One fold, then fresh paint on top.

Each splat is built from a few simple shapes: a body, some lobes, little satellite drops that get smaller the further out they land, and stretched spikes. The idea comes from Lei et al. 2013. Adding noise to the edges lets the shapes run into each other the way real paint does.

If you blend yellow and blue the way most painting apps do, you get grey. The app stores how much light each pigment soaks up (its absorbance), so yellow and blue make green, like they did in school.

Wet paint flows from where it's thick to where it's thin. Right at the edge, any paint thinner than a set grip height stays where it is. That lets a drop round off and come to rest, and it's how the app fakes surface tension. When you squeeze the paint between the sheets, the grip lets go and it spreads again.

When you fold, the paint on both halves meets in the middle. When you rub, it squishes out under the paper.

Under the paper, the squished paint follows the texture of the paper, which breaks it up into veins.

On iPhone Duo the phone is the paper. Close it and rub the outside: the paint soaks through where you press.

When you peel the sheets apart, the paint splits between them in streaks that run from thin paint towards thick. That's what gives you the fingers reaching in from every edge.

Finger directions

Who keeps what

Peel veinsEven split

Veins

The magic straw

Every puff of air breaks up into a few separate streams, and each one drags the thick paint out into a spidery finger.

The air

With streams of air

Without: one smooth tongue

Glitter

The glitter lives in the paint. Every bit of paint remembers where it was first put down and how far it has been pushed since then. Each flake is drawn at that remembered spot and moves with it, so when you smudge glittery paint the flakes stay sharp and travel along with it.

A single shader draws the paint, working up from its height one layer at a time.

All of this runs every frame, and there's still plenty of the frame left over.

A few things in Metal made a big difference. I can chain dozens of small compute passes in a frame and each extra one only costs a few microseconds. Function constants let me compile the flow kernel twice, as a full version and a lighter one for older devices. Mipmaps give me cheap blurs for the darker rims and the peel. Storing paint in half floats halves the memory, and because the CPU and GPU share memory, saving a painting just means reading the paint straight back.