2023 article

Shinyanimalcv: Interactive Web Application for Object Detection and Three-Dimensional Visualization of Animals Using Computer Vision

Wang, J., Xiang, L., Morota, G., Wickens, C., Cushon, E., Brooks, S., & Yu, H. (2023, November 6). JOURNAL OF ANIMAL SCIENCE, Vol. 101, pp. 244–245.

author keywords: computer vision; precision livestock farming; R shiny
TL;DR: The newly developed ShinyAnimalCV is developed, which is a Shiny-based interactive animal computer vision web application that offers a user-friendly graphical user interface for object detection and three-dimensional visualization and could facilitate the application of computer vision in the animal science community. (via Semantic Scholar)
Source: Web Of Science
Added: December 18, 2023

Abstract Precision livestock farming applies integrated sensors and information technology to provide individualized animal care in a timely manner. A computer vision system, a non-intrusive and cost-efficient sensor technology, may be useful for monitoring animals to support farm management decisions. While the advancement in camera sensors provides a great opportunity for producers to improve animal health and welfare sustainably, the limited availability of user-friendly image data processing software tools substantially hinders the implementation of computer vision in livestock production systems. The objective of this study was to develop ShinyAnimalCV, which is a Shiny-based interactive animal computer vision web application. This software tool offers a user-friendly graphical user interface for object detection and three-dimensional visualization. The object detection module employs the Mask-RCNN to precisely segment the masks (regions) of the focal animals from two-dimensional images. A Python-based computer vision library OpenCV was used to draw minimum bounding boxes covering the segmented masks for object detection. The three-dimensional visualization module takes the depth map file captured from a top-view three-dimensional camera as an input, which contains the numerical distances between the camera and the objects (animals and background). The depth map file was first converted to a heatmap image, followed by identifying and segmenting the animal from the background using the Mask-RCNN model. The object detection module returns detection results, including the detected animal’s location, class/type, and morphological traits (e.g., body length and width). The visualization module generates a segmented mask to extract the height of the animal from the depth map file, which can be used to interactively visualize the three-dimensional surface of the animal and estimate its morphological traits, including length, width, height, and volume. By integrating these two modules into R Shiny, we deployed ShinyAnimalCV on a cloud server with pre-trained Mask-RCNN models using pig and cattle data to allow users to upload custom data and perform object detection and three-dimensional surface visualization. The features extracted by ShinyAnimalCV are expected to be useful for performing animal identification, feed intake monitoring, body weight predictions, and body condition score estimations. We conclude that the newly developed ShinyAnimalCV could facilitate the application of computer vision in the animal science community.