Recovering Wind-induced Plant motion in Dense Field Environments via Deep Learning and Multiple Object Tracking

Gibbs, Jonathon A.; Burgess, Alexandra J.; Pound, Michael P.; Pridmore, Tony P.; Murchie, Erik H.

Abstract

Understanding the relationships between local environmental conditions and plant structure and function is critical for both fundamental science and for improving the performance of crops in field settings. Wind-induced plant motion is important in most agricultural systems, yet the complexity of the field environment means that it remained understudied. Despite the ready availability of image sequences showing plant motion, the cultivation of crop plants in dense field stands makes it difficult to detect features and characterize their general movement traits. Here, we present a robust method for characterizing motion in field-grown wheat plants (Triticum aestivum) from time-ordered sequences of red, green and blue (RGB) images. A series of crops and augmentations was applied to a dataset of 290 collected and annotated images of ear tips to increase variation and resolution when training a convolutional neural network. This approach enables wheat ears to be detected in the field without the need for camera calibration or a fixed imaging position. Videos of wheat plants moving in the wind were also collected and split into their component frames. Ear tips were detected using the trained network, then tracked between frames using a probabilistic tracking algorithm to approximate movement. These data can be used to characterize key movement traits, such as periodicity, and obtain more detailed static plant properties to assess plant structure and function in the field. Automated data extraction may be possible for informing lodging models, breeding programmes and linking movement properties to canopy light distributions and dynamic light fluctuation.

Citation

Gibbs, J. A., Burgess, A. J., Pound, M. P., Pridmore, T. P., & Murchie, E. H. (2019). Recovering Wind-induced Plant motion in Dense Field Environments via Deep Learning and Multiple Object Tracking. Plant Physiology, 181, 28-42. https://doi.org/10.1104/pp.19.00141

• Journal Article Type: Article
• Acceptance Date: Jul 9, 2019
• Online Publication Date: Jul 22, 2019
• Publication Date: Sep 30, 2019
• Deposit Date: Jul 15, 2019
• Publicly Available Date: Jan 13, 2020
• Print ISSN: 0032-0889
• Electronic ISSN: 1532-2548
• Publisher: American Society of Plant Biologists
• Peer Reviewed: Peer Reviewed
• Volume: 181
• Pages: 28-42
• DOI: https://doi.org/10.1104/pp.19.00141
• Keywords: Deep learning, Feature detection, Field environment, Image analysis, Movement, Wheat
• Public URL: https://nottingham-repository.worktribe.com/output/2307506
• Publisher URL: http://www.plantphysiol.org/content/181/1/28
• Contract Date: Jul 15, 2019