Kinect MIDI Controller (KMIDIC) – On a Mac!

Instead of making a stop motion video of my giant Lego man (you can see him in the video below) like I was planning to do, I ended up playing with Processing again. Basically I didn’t really have the right space to setup my stop motion, and I really needed to have a green screen to do what I wanted to do. But its all good, no time wasted as I was very productive anyway!

Firstly, heres the video of my creation today:

Now an explanation…

In summary, I am waving my hand around in the air like an idiot and it is controlling the music coming from my computer. The hardware and software components that are at play here are Microsoft Kinect -> Processing -> Ableton Live and Novation Launchpad. I had a chat to my friend DJ Gustavo Bravetti and he had some good tips for me on how to setup Ableton clips to make the transitions smoother and sound more musical. When I have time, I’ll set up a whole song and give a better performance!

A bit more detail…

Basically, the Kinect is sending the location of my hand to Processing which is in turn sending MIDI note on messages to both Ableton Live and the Novation Launchpad. In this version, I have separated the grid into 4 quadrants, each on playing a different MIDI note that is going into 2 channels in live. The first channel has an arpeggiator triggering an Impulse drum kit, and the second channel has an arpeggiated synth. The lights on the Launchpad are also set to light up each of the quadrants as they are triggered.

For those wanting to delve into the code, its not highly commented, but you should be able to get the idea of what I’m doing. Any issues, just leave a message here or send me an email on benxtan [at] gmail [dot] com.

Processing source code and Live set are available here (UPDATED):

http://benxtan.com/kmidic/kmidic_processing_v0.2.zip

http://benxtan.com/kmidic/kmidic_processing_v0.1.zip

You will need to install Processing, and the rwmidi and libfreenect libraries in the libraries folder of your Processing sketches.

Here are some links if you are after more information.

Software Links:
http://www.ableton.com/
http://processing.org/
http://ruinwesen.com/support-files/rwmidi-0.1c.zip
http://ruinwesen.com/support-files/rwmidi/documentation/RWMidi.html
https://github.com/shiffman/libfreenect/tree/master/wrappers/java/processing

Hardware Links:
http://www.xbox.com/kinect
http://www.novationmusic.com/products/midi_controller/launchpad

Once again, if you are a musician or music business in Australia, its free to sign up to http://rockstarhookups.com.au so go do it! I’m giving you free stuff, so help me out here ok? 🙂

Easter Long Creative Weekend – Phase 3: Programming – IN PROGRESS

Digital Art – Where design and visual art meets programming

Having all this free time is great. I finally have a chance to explore and play with things that have been sitting in my todo list, collecting (digital) dust.

Today’s topic of interest is digital/generative art and my weapon of choice is Processing. Check out the links below for some truly beautiful works of art.

Artworks:

• http://www.phidelity.com/blog/blog/design-typography/graffiti-analysis-2-0/
• http://www.complexification.net/gallery/
• http://www.drawnline.net/wisteria/
• http://www.drawnline.net/butterflies/
• http://www.openframeworks.cc/gallery
• http://processing.org/exhibition/

Tools:

• http://processing.org/
• http://www.openframeworks.cc/

After doing some research, I started playing around with Processing. I began messing around with the built-in examples and started a program that uses the mouse speed to draw circles (speed affects radius) and generate a sine wave (speed affects volume and frequency). I soon tired of this and began the task of getting Processing to talk to Ableton and my Novation Launchpad. The idea is that eventually, I can make an aesthetically pleasing visual display that is controlled by a combination of pre-programmed audio coming from Ableton, and a live performance from me using the Launchpad via MIDI, all working together as one synchronised unit.

Heres a screenshot of a work in progress:

What it does:

• Launchpad buttons light up when you press them
• Launchpad buttons light up when pre-programmed drum samples are triggered
• Processing screen lights up exactly the same as what is on the Launchpad
• Processing screen shows what the buttons are mapped to (currently hard-coded…but wouldn’t it be awesome if it was automatic?!)

Sounds pretty simple…but it was a bit of a challenge getting all the settings right! I still have some issues to iron out, and hopefully try and simplify the system more.

I’m very keen to check out Open Frameworks and Max for Live as well. Theres a 30% discount on Max for Live that ends in 4 days, and I’m very very tempted to buy it.

Ableton Live, Processing, Max for Live, Launchpad, iPad, Kinect, Open Frameworks…all wonderful tools to explore and play with. Somewhere in there is just the right combination of hardware and software that will allow me to create something awesome. Something that combines my love of music and programming…

Oh, heres one more link. Its a TED talk on research being done on improvisation (jazz and rap). It doesn’t really reveal much that is new, but its an entertaining talk and opens up some interesting questions. http://www.ted.com/talks/charles_limb_your_brain_on_improv.html