Genelec Helps Turn Movement into Music

Music and movement naturally go together, and there is something primal about the need to dance when we celebrate. It is usually the case that the music leads to the movement, but with the help of technology it can work the other way around. One of the early pioneers of this in the digital arts was Robert Wechsler with his then New York- based dance company Palindrome, which used motion tracking and sensors to trigger sounds, music and visual effects from expressive gestures on stage.

“Triggering a sound through movement is a technical feat but it may or may not make any artistic or intuitive sense,” says Wechsler. “But if you find the right way to do it, you suddenly have this feeling ‘I’m hearing my movement, I made that sound’.”

It is bringing this artistic nuance to what is a highly technical area that Wechsler sees as his specialist field, so it is little wonder that people come to him looking for help with their ideas. One of these interactions in particular has gone on to spawn a bigger project.

“About 10 years ago I was approached by Alicia Peñalba, a Spanish music therapist, with the idea to do motion tracking, making movement into music, for people with disabilities,” he recalls. “She worked with people with cerebral palsy in Valladolid in Spain, and she invited me out for some experiments in that direction and that was the beginning of this project.”

Years of development and workshops followed to refine the concept into a device that would be simple to set up and use, allowing therapists to focus on their work, not the technicalities. The end result, MotionComposer 3.0, is based on passive stereovision – featuring two cameras but no laser or IR projector to understand the positioning of a human body

in a 3D environment. With this information the device can recognise even the smallest movements of the body, such as the blink of an eye, and convert them into a range of sounds.

The reproduction of those sounds is via a pair of white Genelec 8020D active studio loudspeakers.

“The device sits on a table and has two cameras built into it,” furthers Wechsler. “To see shape, you need 3D camera technology which defines what’s a human body, what’s a sofa and what’s a chair. This is quite tricky technically.”

With the device able to recognise movement, the next challenge is to help people understand how their movement can trigger the music. “You need a metaphor,” explains Wechsler. “You could spread piano notes across the room, so if it’s as if you’re dancing on a piano. As you move across the room, you’re going up the scales. You have a clear sense of direction – as I go this way, the notes go higher. Another metaphor might be that the notes are arranged vertically so as you bend low towards the floor, you are playing the low notes and as you stretch up as high as you can, you’re playing the very highest note. But it is tricky to mix these metaphors as the player has to have a sense of what is going on. They could be making music with their movements, but if they don’t have a metaphor or mapping that they can hang onto, you lose the causality.”

Of course, the final piece of the puzzle is being able to hear the music that this movement is creating, and for that the MotionComposer team turned to Genelec’s 8020D active studio loudspeakers. “There were several reasons we chose Genelec,” says Wechsler. “The first is the quality of the sound. Some of the interactive musical environments we put people in are not playing music, but rather natural sounds like rustling leaves or insects. We work with people with very limited movement and so we might say that a very small movement like

a blink of the eye might make a small sound like a twig snapping. So we don’t just want speakers that play music, we want speakers that let you hear and wake your ears up to the full range of what sound can be. We liked the fact that the Genelec speakers really let you hear every little tiny thing that might be there.

“That is also therapeutically important because if you want people to focus on the sense of hearing, one very good way to do it is to get into the world of little delicate sounds and that perks up your aural capacity. The dynamic range is very important, you get the delicacy, but you also get quite a strong kick out of them.”

Equally important was the robust nature of the loudspeakers. “Some of the guys we deal with don’t have great control over their movements, so things might get knocked over,” says Wechsler. “It’s got to be able to withstand some jostling. We liked that the cone is protected and the speaker is solid.”

A third consideration was the physical look of the 8020Ds as MotionComposer needed to appear to be one unified product. “We wanted the machine to be of high quality and we wanted aesthetics to match the design,” notes Wechsler. “The 8020 is white and has the curves and lines, so they look great and match the design. It is an artistic project and we want people to have an aesthetic experience.”

Finally, the social values of Genelec proved the clinching factor. “The company has a good social ethic,” states Wechsler. “I like that the factory is energy neutral. That social ethic belongs to our conscience too. It seems like a good fit.”

With MotionComposer 3.0 hitting the market this Spring, the company is already looking forward to offering more to its users. “Once we prove our concept to the world, then we have lots of ways we want to expand,” concludes Wechsler. “We’d like to expand it musically and in terms of features. We believe that people with other abilities should get good stuff. They might not be able to move much so it becomes even more important if you are limited in some ways that you have access to the best kind of experience.”