Computer vision to expand monitoring and accelerate assessment of coastal fish (CoastVision)

Funding agency: The Research Council of Norway, Researcher Project for Scientific Renewal, Oceans Program
Project period: 2021-2025
Project leader: Kim Halvorsen
Project participants: Tonje Knutsen Sørdalen, Morten Goodwin, Diana Perry, Suzanne Alonzo, Cigdem Beyan, Holly Kindsvater, Howard Browman, Anne Berit Skiftesvik, Vaneeda Allken, Ketil Malde, Kristian Knausgård

Project summary
Effective monitoring and management of coastal ecosystems is limited by observation methods. Underwater cameras are increasingly being used to monitor and study coastal fish communities; a major bottleneck for upscaling their use is dependence on human experts for image and video analysis. CoastVision will use the power of deep learning to refine and extend a computer vision pipeline for detecting, classifying and sizing the key fish species in shallow water coastal ecosystems, facilitating a transition to fully automated video analysis. Our models will be trained on data sets from several different surveys, ensuring cost-efficient development of routines that will be widely applicable. Computer vision for re-identifying (re-ID) individuals solely based on their unique visible features will also be developed. This novel aspect of CoastVision could ultimately provide new opportunities to obtain detailed knowledge about behaviour and population dynamics in wild fish populations, with minimal negative impact on animals and habitats and at a low cost. Our focal species for re-ID are Atlantic cod, ballan wrasse and corkwing wrasse, commercially important species with complex, high-contrast skin patterns. To generate the necessary training data for re-ID we will use synchronized radio frequency identification and camera systems. CoastVision’s automated video analysis pipeline will be integrated into ongoing ecosystem surveys and case studies whose main objective is to better understand the factors that affects the reproduction, recruitment and survival of commercially important coastal species. As such, CoastVision will contribute to independent, but complementary, research objectives. The project will advance the international research front for applied machine learning in marine ecology, which ultimately can revolutionize our ability to observe, understand and respond to ecological change at scales far more refined than is currently possible.