Multiclass insect counting through deep learning-based density maps estimation

Abstract

The use of digital technologies and artificial intelligence techniques for the automation of some visual assessment processes in agriculture is currently a reality. Image-based, and recently deep learning-based systems are being used in several applications. Main challenge of these applications is to achieve a correct performance in real field conditions over images that are usually acquired with mobile devices and thus offer limited quality. Plagues control is a problem to be tackled in the field. Pest management strategies relies on the identification of the level of infestation. This degree of infestation is established through a counting task manually done by the field researcher so far. Current models were not able to appropriately count due to the small size of the insects and on the last year we presented a density map based algorithm that superseded state of the art methods for a single insect type. In this paper, we extend previous work into a multiclass and multi-stadia approach. Concretely, the proposed algorithm has been tested in two use cases: on the one hand, it counts five different types of adult individuals over multiple crop leaves; and on the other hand, it identifies four different stages for immatures over 2-cm leaf disks. In these leaf disks, some of the species are in different stadia being some of them micron size and difficult to be identified even for the non-expert user. The proposed method achieves good results in both cases. The model for counting adult insects in a leaf achieves a RMSE ranging from 0.89 to 4.47, MAE ranging from 0.40 to 2.15, and R2 ranging from 0.86 to 0.91 for 4 different species in its adult phase (BEMITA, FRANOC, MYZUPE and APHIGO) that may appear together in the same leaf. Besides, for FRANOC, two stadia nymphs and adults are considered. The model developed for counting BEMITA immatures in 2-cm disks obtains R2 values up to 0.98 for big nymphs. This solution was embedded in a docker and can be accessed through an app via REST service in mobile devices. It has been tested in the wild under real conditions in different locations worldwide and over 14 different crops.