A Successful Crowdsourcing Approach for Bird Sound Classification

Abstract

Automated recorders are increasingly used in remote sensing of wildlife, yet automated methods of processing the audio remains challenging. Identifying animal sounds with machine learning provides a solution, but optimizing the models requires annotated training data. Producing such data can require much manual effort, which could be alleviated by engaging masses to contribute to research and share the workload. Birdwatchers are experts on identifying bird vocalizations and form an ideal focal audience for a citizen science project aiming for the required multitudes of annotated avian audio data. For this purpose, we launched a web portal that was targeted and advertised to Finnish birdwatchers. The users were asked to complete two kinds of tasks: 1) classify if a given bird sound belonged to the focal species and 2) classify all the bird species vocalizing in 10-second audio clips. In less than a year, the portal achieved annotations for 244,300 bird sounds and 5,358 clips, and attracted, on average, 70 visitors on daily basis. More than 200 birdwatchers took part in the classification tasks, of which 17 and 4 most dedicated users produced over half of the sound and clip classifications, respectively. As expected of birder experts, the classifications among users were highly consistent (mean agreement scores between 0.85–0.95, depending on the audio type) and resulted in high-quality training data for parameterizing machine learning models. Feedback about the web portal suggested that additional functionality such as increased freedom of choice would increase user motivation and dedication.