A Dual-Stream Deep Learning Architecture for Action Recognition in Salmon from Underwater Video.
MetadataShow full item record
Over half of the costs from breeding salmon in the Norwegian salmon farmingindustry comes from feed usage. Today the feeding process is largely a manuallabor, requiring an operator to monitor the amount of feed sinking to the bottomof a breeding cage. When the amount of feed exceeds a certain threshold, thefeeding process is terminated. Automation of this process and using salmon motionbehavior instead of sinking feed to determine when to terminate feeding, couldgreatly reduce costs both in through the labor needed and amount of feed wasted.Resent developments in Human Action Recognition have shown that DeepLearning approaches are well suited to perform Action Recognition. Wetherefore examine the feasibility of using Deep Learning approaches to automate thefeeding process. We use 76 videos of salmon collected from within a breeding cageduring the month of November 2016. Using these videos for training, validationand testing, we propose three approaches to automatically classify Feeding andNonFeeding behavior in salmon. The three approaches are a Spatial Architecture, aSpatial Recurrent Architecture and a Dual-Stream Architecture.Our results show that all our proposed architectures are able to separate videosof Feeding and NonFeeding salmon with high accuracy. We also find that the Dual-Stream Architecture is the best performing architecture. It combines spatial andtemporal information through the use of a Spatial Stream, a novel Temporal Streamand a Recurrent Neural Network (RNN). Our Dual-Stream Architecture is ableto accurately classify 80.0% all of our testing videos, presenting state-of-the-artperformance.To the best of our knowledge, both our Temporal Stream and our Dual-StreamArchitecture are original and novel architectures, as is the application of Deep Learning inference models for the Salmon Activity Recognition domain in optimizationof feeding operation in Norwegian Aquaculture. We hope the results presented inthis thesis will contribute to achieve a higher sustainability in Norwegian salmonaquaculture, optimize feeding operations, and consequently reduce potential waste.Future work beyond the results presented in this thesis concerns research on understanding of what our the Deep Learning architecture have learned and visualizingthis learning process.