DEEP LEARNING FISH IN TANK BEHAVIOR MONITORING

Introduction

 Fish behavior activity is complex activity, which is traditionally determined by human experts, telemetric tags, flow through chamber of by tracking of the individual fish using computer vision. Several methods of fish behavior monitoring have been developed for different reasons.  The most promising and cheap method is computer vision fish tracking. It provides an automated, non-invasive, and cost-effective methods of recording behavioral parameters. Single camera or multi-camera 3D machine vision has been used in behavior analysis or trail tracking (Saberioon and Cisar, 2016). The visible or near infrared light is used for fish monitoring (Farokhi et al., 2016, Pautsina et al., 2015). The output of the monitoring systems is the 3D trajectory of the fish which represent the fish swimming during the given period. The main issue of the monitoring systems is the tracking of the fish in the tanks with high fish density. The fish overlapping enables to monitor individual fish behavior just for very limited time. Therefore, some studies focus on the parametrization of the behavior of the fish school instead of the individuals.

Application of the modern deep learning methods (Nichols et al., 2019) could solve the problem of fish behavior parametrization without the need to detect and track individual fish. To be able to optimize the feeding we have to distinguish (classify) different states of the fish based on the analysis of the behavior. The deep learning concept will be used to automatically parametrize fish behavior and classify it into predefined classes. The research is focused on the feeding optimization in intensive aquaculture and specially on the recirculation aquaculture systems (RAS) represented by the fish cultivation tanks. This focus is given by the possibility to control all conditions of the fish cultivation and precise control of fish feeding and feed distribution in the tank.

Materials and methods

 The initial experiments were focused on the classification of Sea bass behavior into 3 classes: normal, stress (caused by knocking on the tanks) and feeding, Fig. 1. The video records of fish swimming in the standard cultivation tanks (3 tanks) were collected . The short videos (4 minutes ) of all three types of behavior were collected using 45 fish.  V ideo data  were cut into 3 seconds clips and resized to 228 by 228 pixels. Only the inner area of fish tank was used for processing. Totally 210 sequencies were obtained for each class and 100 randomly selected sequencies were used for classification validation. for The GoogleNet feature extractor (Szegedy, 2015) was used for data processing.

Each frame of video was transformed into 1024 features and stored in vectors in sequence of 45 rows (3 second of 15 fps video). After preprocessing, feature vectors sequencies were feed to L ong S hort-Term Memory (LSTM) network to process the sequential data and perform the classification.

.

Results

The classification accuracy after 30 epochs of training  was 99 %, such high accuracy shows that  classes are significantly different and easily distinguishable by the network. On validation dataset accuracy was 99% that also show that our network was not overfeed.

 Discussion

 In this study we proved that we could distinguish between different types of  fish behavior using the concept based on deep learning. The concept avoids the detection and tracking of individual fish in the tank.  The disadvantage of the approach is generalization of the solution. The system is specific for the fish species and for the cultivation tanks used for network training. The similarity of cultivation tanks (mainly circular shape) and possibility to make specific models for fish species will enable to use the concept for general fish behavior monitoring. We started new experiments for common carp with more classes of fish behavior to study the possibility of feeding optimization and fish welfare determination.

 References

Farokhi et al., 2016, S. Farokhi , J. Flusser, U. Ullah Sheikh, Near infrared face recognition: a literature survey, Comp. Sci. Rev., 21 (2016), pp. 1-1

 Pautsina et al., 2015, A. Pautsina, P. Císař, D. Štys , B.F. Terjesen , Å.M.O. Espmark , Infrared reflection system for indoor 3D tracking of fish, Aquacult. Eng., 69 (2015), pp. 7-17

 Saberioon and Cisar, 2016, M.M. Saberioon, P. Cisar, Automated multiple fish tracking in three-dimension using a structured light sensor, Comp. Electron. Agricult., 121 (2016), pp. 215-221

 Nichols, J.A., Herbert Chan, H.W. & Baker, M.A.B. Machine learning: applications of artificial intelligence to imaging and diagnosis. Biophys Rev 11, 111–118 (2019). https://doi.org/10.1007/s12551-018-0449-9

Szegedy et al., 2015, W. Liu, Y. Jia Y., "Going deeper with convolutions." In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1-9. 2015.