Show HN: Duetto – pretrained music transformer that plays a piano with you

duo-f-blues.mp4

Play a MIDI keyboard and a transformer plays a second piano with you, in real time. It listens like a partner: your key, the chord you play, your pulse, your phrases. It comps while you play and takes the melody when you rest.

The model is the pretrained Anticipatory Music Transformer (Lakh MIDI), running on a Jetson; the laptop handles MIDI, sound and the piano roll.

duo-you-comp-it-solos.mp4

Above: you play low chords, the model improvises over them (the it solos preset). Files in samples/duo.

ssh -t jetson sudo jetson_clocks # after every Jetson reboot ./run_duet.sh # on the laptop

run_duet.sh restarts everything: the model server on the Jetson, the SSH tunnel, FluidSynth, main.py on the keyboard and the piano roll. Closing the piano roll stops the duo and saves it in generated/.

Without a Jetson: python -m music_transformer.server in one terminal, python main.py in another (Ctrl+C stops and saves).

The top of the piano roll says what the model hears (key, chord, tempo) and what it is doing. Keys: F fullscreen, K show or hide the controls, Esc close.

Controls are sliders (drag them; hold Shift to drag finely) and steppers (click the arrows). The wheel or the arrow keys over a control turn it, and the wheel elsewhere scrolls the panel. A double click resets a control, and the bottom of the panel says what the control under the mouse does. A control is dimmed while what it shapes is not happening.

Knobs and sound are kept in ~/.config/ai-duet/.

PolyForm Noncommercial 1.0.0: use it, change it and share it freely for anything noncommercial (personal, study, research, education, hobby). Commercial use is not allowed. The model itself is the Anticipatory Music Transformer, under its own license.