Adam Behrman
Class of 2021
- Computer Science
Developing a Mobile Application to Facilitate Aural Training and Synesthetic Experiences
Project Mentor:
- Duke Hutchings, professor of computer science and chair of the Department of Computer Science
Project Abstract
Music learners around the world are eager to improve their skills, but some may struggle to find a teacher or an instrument. Additionally, current teaching methods for certain skills, such as relative pitch (RP), may be ineffective at best. An alternative currently being proposed to these methods is using sound/color associations to improve music recognition. Can a mobile app be used to facilitate this training? We will introduce Chroma, a mobile application designed for the purpose of developing chromesthesia (that is, a neurological condition in which sounds involuntarily and consistently produce a visual response) in music learners through musical note identification tests and an augmented reality (AR) visualizer. Past and current methods of teaching synesthesia and commonly-taught aural skills such as relative pitch (RP) will be evaluated. This study also addresses the “unteachability” of absolute pitch (AP) and explores its relationship to synesthesia. The benefits of mobile applications for learning are discussed, as well as the use of AR for visualizing chromesthesia via a novel multiple pitch detection algorithm. The authors also introduce a new test combining a traditional test for AP and a training program to develop chromesthesia into one simple pitch identification quiz. The sample for this study is Aural Skills 1 students at Elon University. The authors hypothesize that students who engage with the app as a part of regular piano training will: 1) Develop synesthetic experiences, and 2) improve their musical skills faster than those who do not have access to the app. In order to improve the functionality and effectiveness of the training, the authors will collect data based on students’ performances on pitch identification tests over the course of one semester (roughly 12 weeks). At the end of the test period, an assessment will be conducted to determine the effects of the app and overall user satisfaction.