5-23-11

Student research group learned teamwork in electronic music course that fit like a glove

Christopher Hopkins (left), associate professor of music, wears a data glove and controls the sound and images. Assisting him is Fernando Bravo, a member of the graduate-level seminar who is ISU's first graduate of the Integrated Electronic Arts cluster

At first glance Christopher Hopkins appeared to be playing a video game by moving his gloved right hand. A melodic sound emitted from speakers and colorful shapes gently moved across a large video screen.

The glove is no ordinary one, said Hopkins, associate professor of music in Iowa State University's Department of Music and Theatre. And it's not a video plaything, but a scientific tool used by Hopkins and four student researchers who have combined the arts and technology.

Hopkins wore an AcceleGlove, a high-tech glove equipped with six accelerometers, devices that translate motion and positions of the hand, wrist and fingers to data for input to a computer.

Hopkins and his students (three of whom graduated in May 2011) used the glove in a graduate-level seminar that focused on designing interfaces and software for performance of electronic music. The students, whose academic backgrounds include music, psychology, engineering and graphic design, spent two years together in the course.

"Two of the students had collaborated during the first year to design a musical glove controller from the ground up, so I suggested this as a focus for continuing the course involving all four students in team-based research," said Hopkins.

Electronic gloves have been used as music controllers since the 1990s, but those mostly used flex sensors, which measured just the bending of fingers. The AcceleGlove, made by AnthroTronix Inc. of Silver Spring, Md., uses three-axis accelerometers on each finger and the palm, which can capture changes of speed and direction.

"This kind of data glove transmits data for finer changes of motion in all three dimensions," said Hopkins.

The AcceleGlove data glove.

Vibrational feedback

He believes the most significant contribution of the team's work was to add "vibrational feedback" to the glove.

Hopkins said, "While the Acceleglove on its own sends information, our augmentation allows the performer also to receive information as vibration on the fingers. Music is a language built on vibration, so we are in a good position to create a kind of language of vibrations, one that is more subtle and flexible than, for example, Morse Code."

Software developed by the team for the glove allows users to move their hands and fingers to change the speed and tone of both music and video, mapping rhythmic patterns and even drawing these in colors on the screen.

"It's somewhat like conducting," Hopkins said. However, he is not looking at the project as a way to conduct an artificial orchestra. "There's no substitute for the real thing. We conduct the real-time construction of the music. It is a hybrid of composing and performing."

Hopkins said the students' varied skills all contributed to the effort:

• Fernando Bravo, a musician and psychologist, worked with 3D modeling and motion graphics. He completed an M.S. degree in Interdisciplinary Graduate Studies and is ISU's first graduate of the Integrated Electronic Arts cluster. This fall he begins a Ph.D. program at Cambridge University in England.
• Karl Svec provided computer and electrical engineering skills. He remains involved in the virtual sound and music control project headed by Hopkins and is completing a master's degree in human-computer interaction.
• Jeffrey Kosinski contributed music programming and graphic design knowledge. He recently earned a graphic design degree.
• Mark Weimer built the wireless communications and collaborated with Svec on programming the glove's vibration controls. He has a mechanical engineering degree and a minor in music technology.

Sharing discoveries

"This course featured modular, team-based research that connects the arts and technology," Hopkins said. He added that the project taught students to work as a team toward a common goal.

"This allowed them to experiment and prepared them for a research and working environment that's increasingly team-based. They learned a lot about teamwork and ways of sharing discoveries and solutions with other researchers."

Beyond the technological outcomes, Hopkins believes the team's collaborative work will help them in years come.

"Really, this is where I envision the value of this kind of educational experience, that students may realize the value of pursuing a collaborative creative complement to their future roles in industry and research.

"Ultimately the success of this venture may be found in the years ahead."

Although the team members are moving on to different ventures, they plan to continue the collaboration in some form.