Abstract
Tobias Grosshauser; Thomas Hermann
Motion and gesture are important parameters in musical instrument playing and examined extensively in many studies and research projects. But additionally pressure and force are important parameters, but still very little research is done in this area. Pressure sensing technologies allow to measure and interpret complex and hidden forces, which appear during playing a musical instrument. The combination of our new sensor setups with pattern recognition techniques like the lately developed ordered means models allows fast and precise recognition of highly skilled playing techniques. This includes left and right hand analysis as well as a combination of both. In this talk we show bow position recognition for string instruments by means of support vector regression machines on the right hand finger pressure, as well as bowing recognition and inaccurate playing detection with ordered means models adapted to individual students. We also show a new left hand and shoulder/chin sensing method for coordination and position change analysis. Our methods in combination with our audio, video, and gesture recording software can be used for teaching and exercising. Especially studies of complex movements and finger force distribution changes can benefit from such an approach. Beside the sensing part of also the feedback part is examined. Beside visual feedback, audio and haptic feedback methods are tested and evaluated. Additional audio as well as haptic feedback closes the loop from sensor signals over data manipulation to the user in realtime and postprocessing scenarios. In general, all data allow precise off-line examination and comparision for advanced gesture and motion studies. Practical applications include the recognition of inaccuracy, cramping, or malposition, and, last but not least, the development of augmented instruments and new possibilities for modern music.