A playful installation that turns every-day objects into musical instruments.

After the pandemic seemed to cool down a bit during the summer of 2022, Yasha Jain and I wanted to bring back the fun of meeting random people. To easen up the atmosphere we planned to install artworks at a party in her apartment. Besides a touch-sensitive AV installation on a plant, Yasha developed a smile-detecting mirror and a kinect-based projection. robotcowboy contributed an overwhelming performance that got people dancing for the first time in a long while. In a seperate room COCOLOCO was installed, a playful installation that turned every-day objects into musical instruments.

Setup

On a desktop table every-day objects like a scissor, plant, banana and a book are distributed. A webcam hanging from the top is looking at those objects and its image is displayed on a screen. The camera is hooked-up to a Raspberry Pi which in-turn classifies objects in the image using YOLO v4 and sends out information over OSC. A pure data patch reacts to the incoming data and synthesizes different sounds. That is, each object class is associated to an instrument which can be on and off. By moving the object on the 2D plane the user may vary the rythmic pattern and/or modify a sound quality such as filter cutoff or pitch.

Credits: Yasha Jain (Embedded YOLO acceleration)