Xiaoli Fern and Raviv Raich, both associate professors in the School of Electrical Engineering and Computer Science, listen intently to bird recordings from the H.J. Andrews Experimental Forest in the McKenzie River Watershed.
“Is that a warbler?” Fern asks.
It’s hard to tell, given the overlapping calls and noisy stream in the background. But through sophisticated signal processing and machine learning techniques, Fern and Raich are decoding complex recordings to identify different avian species.
Because birds can be a good indicator of biodiversity, Fern and Raich’s research will help scientists understand bird behavior and provide a clearer picture of environmental impacts such as habitat loss and climate change. They are also some of the first to develop computer-based methods to decipher multiple, simultaneous bird calls from different species in the wild, which earned them a mention in Forbes’ list of six novel machine learning applications.
Fern and Raich’s research is partially funded by the National Science Foundation’s Integrative Graduate Education and Research Traineeship (IGERT), a program that promotes the education of engineers and scientists through collaborative and interdisciplinary research.
“Oregon State has strength in computer science, mathematics, and also in ecological science. The initiative explores how we can bring these together,” Fern said.
Although Raich and Fern aren’t birders themselves, they saw machine learning as a powerful tool to aid the ecological sciences and support the IGERT program. Both are NSF Faculty Early Career Development (CAREER) Award winners and have assigned the project as one of the key application areas in their CAREER research.
“Machine learning deals with analysis of data for many different disciplines,” Raich said. “We tend to think of analysis of websites or a text document. But pretty much anything can be analyzed using machine learning.”
Traditional methods for collecting bioacoustics data on birds have focused on clean recordings of a single call. “That wasn’t an option for us,” Fern said. “Our recordings had lots of noise — wind, a stream, lots of birds, and rain.”
Working with the raw files, Raich and Fern first cleaned up background noise and converted the files to what are called spectrograms. A spectrogram, like a musical score, provides a visual representation of sound over time. It allows researchers to better analyze data.
“If you looked at just the raw signals, it’s impossible,” Raich said. “It would be too messy to analyze.”
Their next challenge was how to process and label terabytes of bird recordings in an efficient and accurate way.
In comes machine learning. Using a technique called multi-instance multi-label learning, Fern and Raich developed algorithms that learn to identify and differentiate vocalizations of different species with limited human "supervision" from expert birders.
“We don’t want to make the human expert delineate each and every sound,” Fern said. “That would take too long and prove too difficult.” Instead, bird experts provide a broad summary of what species they hear, which gives the computer a baseline of examples to make further associations. “Through sophisticated machine learning techniques, we can in essence ‘teach’ the computer to disambiguate the sounds.”
Though the project is ongoing, Raich and Fern are seeing results. In a study published in the Journal of the Acoustical Society of America, they evaluated 548 10-second recordings containing 13 species. Despite the noisy, multi-bird recordings in the field, their methods accurately predicted the set of species present.
“We’ve produced some tools that allow us to analyze recordings collected in the field with multiple bird species singing on top of each other. We can handle that kind of recording and recognize which species are there with reasonable accuracy. That wasn’t done before,” Fern said. “We’re hoping to use that information to get a sense of activity level and what variables influence bird behavior.”
In addition to publishing their findings, Fern and Raich submitted hundreds of 10-second audio clips gathered at the H.J. Andrews long-term ecological research site to Kaggle, a competitive data analytics website. The submission provided a platform for researchers across the world to put their best efforts toward the bird bioacoustics problem. By doing so, Fern and Raich hope to help advance both machine learning and bird bioacoustics research through community engagement.
“The motivation for us was getting the data out there and involving the broader research community,” Raich said.
Using audio recordings and computers to identify birds has several advantages over human surveys, including less observer bias, reduced costs, and access to remote sites that give scientists a clearer picture of population dynamics.
But Fern and Raich stress that they are not trying to take the human element out of bird research. It still takes people to decipher the ecological meanings of the data.
“We want to bring together people and technology,” Raich said.
Xiaoli Fern and Raviv Raich would like to acknowledge a few key colleagues involved in the bird research: Matt Betts, College of Forestry; Dave Melligner, Hatfield Marine Science Center; and Jed Irvine, School of Electrical Engineering and Computer Science.
— Abby P. Metzger