Student: Josh Mao
Course: DES INV 23, Creative Programming & Electronics
Chromesthesia, or sound-to-color synesthesia, is a type of synesthesia in which sound involuntarily evokes an experience of color, shape, and movement. Many piano players, including Josh Mao, have hints of this condition, but rarely find it easy to access the multitude of colors that a piano piece can create while being played. Curious to explore how to see what a piano sounds like, Josh created a fully functional, piano visualizer that utilizes LEDs to create artificial chromesthesia.
To do this, he hooked up a microphone input to Processing, and an Arduino to a strip of LEDs, designing and engineering an interactive auditory and visual experience that can be enjoyed by professional players and beginners alike. The visualizer pairs LED brightness with the volume of the piano, LED color with the pitch of the piano, and LED pattern with the complexity of the chords played on the piano. It utilizes the R2D2 Processing Pitch program for FFT analysis written by L. Anton-Canalis and builds on top of it to send values over serial to the Arduino, which will then control the LED strip. It’s best enjoyed in the dark, where the vibrancy of the colors can really come to life.
Josh was assisted on this project by Jacobs Design Specialist, Chris Parcell, in addition to Tina Piracci and Professor Michael Shiloh.
