Jordan Rudess talks about his new HarmonyWiz app
Watching keyboard master Jordan Rudess perform his adventurous, virtuosic parts on stage with Dream Theater, you'd swear that he possessed an extra hand or two. Off stage, he's no less of a multi-tasker: As president of Wizdom Music, he's designed and has overseen the launch of a series of highly successful music-making apps such as MorphWiz, SampleWiz and GeoSynth.
His latest creation, HarmonyWiz, the recently released HarmonyWiz, looks like another winner. Rudess worked with developer Ruben Zilibowitz to formulate an app that would allow users to paint a symphony and orchestrate their music in one touch, using either their finger or by inputting a harmony line from a piano-style keyboard. HarmonyWiz can be purchased and downloaded via iTunes.
In the interview below, Rudess talks about how HarmonyWiz came together and discusses its various functions.
Have you worked with Ruben before?
“No, this is my first time working with him. With my company, Wizdom Music, I work with designers from all over the world. I’ll find someone who’s working in an area of music that’s interesting to me musically and who works with multi-touch devices, and we’ll start up a relationship and work on a project. There’s one developer I’ve done a lot of things with, the fellow I started out with – his name is Kevin Chartier, a very talented guy; we’ve done MorphWiz and SampleWiz, and he also helped out with GeoSynthesizer – so he and I have a lot of history together. But I’ve had some successful and interesting collaborations with other guys, so that keeps it fun.”
What was it about Ruben that intrigued you?
“I have my eyes peeled on the App Store for all the different and cool things happening. Besides at this point being just a jungle of apps, it’s still a place where there’s a lot of creativity. I’m always excited when I see people trying to push the boundaries, whether it’s doing something that’s wild and crazy or something just plain functional and handy.
“In Ruben’s case, I saw a simple app that was playing with an idea that I’d had, one that could analyze a composer or an improviser’s brain. The technology would allow the computer to make decisions about what kinds of chords to use in a musical phrase or piece. What I was imagining was that I could put in a single melody line, hit a button and the computer would say, ‘OK, in this style, here’s a nice little arrangement.’"
Painting being converted to music notation
“Computers work with a set of rules, so if you tell them what you want them to do and give them the rules, they can do a great job. When sitting at my piano and I’m writing or improvising, or if I’m writing a piece for dream Theater, there’s a window of possibilities. If I go into the jazz-chord thing, that’s not necessarily going to work, but if I stay within a particular harmonic structure, those ideas will work. Even in my own brain, I’m going for a set of parameters for a composition, so I thought, ‘That’s something I could tell a computer how to do.’
“Working with Ruben was my start to make this dream come true. What’s amazing and encouraging is that we were able to produce an app, HarmonyWiz, that when you put in a single musical line by painting with your finger, or by inputting it with an onboard keyboard or a MIDI keyboard, or even by importing a MIDI file, it can create really good-sounding multi-part arrangements of what you put in as a single line. It’s pretty crazy.
“I should make clear to people that this kind of work takes a long time. The coding foundation of HarmonyWiz is something that Ruben’s been doing for years. We worked on it for at least a year, just to get it to where it could take classical harmony and do a kind of intelligent and musical job of offering a nice classical/baroque-type harmony to the input phrase. If somebody says, ‘Can it do that for jazz?’ Give me a year and I can go back with Ruben to make new rules for that as well.”
Selection menu
How will a musician actually use HarmonyWiz?
“We developed it for a wide audience. For a guy like me, a professional musicians, it can just make me smile. I’ll sit there and I’ll put in a melody, and I’m fascinated by what it comes up with – it’s not gonna play what I necessarily would have. Or it might, and I go, ‘Holy shit! This app that I created just played the same chords that I would have played.’ In a way, it takes a bit of the mystery out of it, but it puts mystery into it as well. It’s exciting. It offers a different take on a melody that I have in my head.
“It’s very quick too. In the basic root level, all you do is play the melody, hit the button and it’s done. And the cool thing for a musician or a student is that once the melody is in there and the harmony gets notated right in front of you, which you can see very clearly, it shows you what we call the Figured Bass below. ‘Figured Bass’ is the theoretical analysis of chords. If a music student wants to know ‘What is it about this classical harmony that works?’ not only can he see the notes, but he can also see the theory behind it, which is cool.
“But you know, it’s also for somebody who is a non-musician or who is just leaning. Like I said, it’s the kind of thing that can just make you smile. I can put HarmonyWiz in front of somebody, they can draw a really pretty line with their finger, hit the button, and the next thing you know, there’s a piece. To me, that’s just adding joy to somebody’s life.”
You said that you’re surprised sometimes by what HarmonyWiz gives you back, but can you tweak it to give you something you want?
“Yes, absolutely. It’s got a bunch of tools, like a pencil tool so you can write in your own notes if you don’t like what’s there. You can select portions of it, copy and paste. You can say, ‘HarmonyWiz decided to harmonize this note but not that note,’ so you can switch them around if you like. There’s a lot of flexibility to it. You can add measures if you want. It allows you to have five different parts, five different instruments.”
Partially completed arrangement
In the notes, you said that you thought it would be fascinating to capture a songwriter’s thought process in code. To a layman, what does that mean?
“Basically, it’s a set of rules. HarmonyWiz is something like 40,000 lines of code. Somewhere, either a computer or a person has to decide which moves to make. If I hit a C major chord in the key of C and my melody goes to B, there’s certain choices there. Or if you go to a B, chances are you don’t want to go to an F chord around it because it’ll create a dissonance. So if I did hit a B, in my own mind I’m saying, ‘OK, I’m in C major, I hit a B – probably the most common chord would be a G chord,’ which would be the V chord. But I might want to go to an E minor chord, and if I do go to a IV chord, I’ll have the B be a passing tone and I’ll resolve it. So we literally set up all of those rules. Things like that say, ‘OK, you’re on this melody note, what are the possible chords?’
“But it’s also looking in time. ‘What chord did you come from? What chord are you going to?’ There’s harmonic rules that make up the style I know in my head – and musicians who have studied know as well – that you can tell the computer program to follow. Maybe people have heard ‘Don’t have parallel fifths,’ which you wouldn’t have in strict baroque harmony. HarmonyWiz, in the strictest style, won’t allow that, so it’ll find other options.”
HarmonyWiz in the process of arranging a song
What kind of trial-and-error process is there to designing an app? Do you guys go back and forth making tiny tweaks before you’re happy with it?
“It’s a crazy, crazy process. From my position as a designer and musician, it involves a lot of conversation with Ruben – I tell him what I want and I make various comments. If I try the app and I think it’s harmonizing too often – ‘Why don’t we wait and not harmonize on the off-beats but harmonize on the main beats?’ – then I try to consider what we should do about it. It’s about looking for what choice is more musical.
“One of the interesting things I did was play a melody, like, 30 different ways. I’d start out and I’d play it in a classical harmony, and I’d see how many interesting things I could do staying in that mode. Then I stretched it out and loosened it up, and I sent out to Ruben and we’d talk about it. He’d put things into code, and I’d give him feedback.
“Right now with HarmonyWiz, you have five different variations of styles. It starts out with Baroque, which is very serious classical harmonic style, but you can open it up and loosen it up and have different choices. There’s so much to it – I really think it provides a pretty limitless array of choices. You’re not likely to exhaust what it can do – put it that way.”
