Low-Cost Cameras Could Be Sensors to Remotely Monitor Crop Stress

By Mollie Rappe

Being able to identify crop problems early can make the difference between saving a crop and losing it, but high-tech solutions can be costly. An interdisciplinary team of researchers thinks a new approach leveraging existing technology may be part of the solution.

Specifically, NC State researchers in the Department of Crop and Soil Sciences and the Department of Electrical and Computer Engineering are launching an inexpensive camera system that can monitor crop stress remotely.

Corn and soybeans are important commodities for North Carolina and the world. Both are eaten fresh, processed into a variety of foodstuffs and turned into animal feed. A lack of water at certain stages stress the plants, and can make a significant dent on yields.

Paula Ramos-Giraldo, a computer vision and machine learning expert in the Department of Crop and Soil Sciences, has spent the past year working on a camera system that costs less than an average smartwatch to track drought stress in corn and soybean fields.

“Our goal, specifically, was to construct a low-cost sensor to track the soil moisture level in the field through plant behavior,” Ramos-Giraldo said.

These low-cost sensors can help researchers studying ways to make agricultural systems more resilient; plant breeders breeding more drought-tolerant varieties; and some day may be able to alert farmers when their fields need to be irrigated.

Constructing a StressCam

The StressCam system — constructed from parts that cost about $150 — is based around a Raspberry Pi. A Raspberry Pi is a tiny, inexpensive and easily-programed computer originally designed for teaching computer science.

The tiny WiFi-enabled computer includes a camera for taking pictures of a field and is hooked up to a timer that turns the system on in the morning and off in the evening, Ramos-Giraldo said. For corn, the camera is mounted at a 90-degree angle over the field and takes photos every 30 minutes to watch for curling leaves. For soybeans, the camera is mounted at a 45-degree angle over the field and takes photos every 15 minutes to watch for wilting. The system is solar powered, with a back-up battery for cloudy days.

The tiny computer runs a machine learning algorithm on the photos to analyze them for indications of drought stress. Then it sends this information to a web platform for researchers, breeders or farmers, she said.

Both the machine learning algorithm and the web platform were constructed with help from students in the Department of Electrical and Computer Engineering.

Paula Ramos-Giraldo, left; Anna Locke, center; and Edgar Lobaton, right; have worked together to launch an inexpensive camera system to monitor crop stress remotely. (Photo from Summer 2019)

Teaching a Machine to Score Stress

During the fall 2019 semester, Ramos-Giraldo worked with Edgar Lobaton, an associate professor in the Department of Electrical and Computer Engineering, to enlist the students in his Neural Networks class to design machine learning algorithms capable of looking at photos of soybean fields and score the severity of drought stress, she said.

Machine learning algorithms can find patterns in data without being explicitly programmed what important features to look for. Instead, they are “trained” on pre-defined data – in this case 5,000 photos of soybean fields showing different amounts of drought stress annotated by Anna Locke, a U.S. Department of Agriculture- Agricultural Research Service (USDA-ARS) soybean expert in the Department of Crop and Soil Sciences.

The class’s algorithms were then tested on thousands of other photos taken by the StressCam during the summer of 2019. One of the best algorithms was programmed into the StressCam.

In the fall of 2020, Lobaton will run another competition in his Neural Networks class to try to develop more efficient algorithms, such as ones that require less computer memory and thus would run better on the tiny Raspberry Pi.

Seniors Design Web Platform, Despite Pandemic

Also during the fall of 2019, Ramos-Giraldo started working with a team of senior Electrical and Computer Engineering students to design a cloud-based web platform to allow farmers, soybean breeders and researchers to manage their StressCams and monitor their fields.

This team included Artem Minin, Nathan Libner, Stephanie Sierra and Manish Goud.

“It must have taken us the first three weeks to get our heads wrapped around how we could build a platform to solve the problem that Paula Ramos-Giraldo had,” said Minin, who will be returning to NC State in the fall to pursue a master’s degree in computer engineering. “That was one of the really difficult parts of the project because none of us had ever really built such a diverse system with so many different components, technologies, and requirements.”

The web platform allows the users to check that the StressCam is on and not overheating, change the photo scheduling and importantly, look at the past images from the StressCam and the drought stress severity scores. Overall, the platform will save time and increase the precision of drought data collection for plant breeders and researchers studying resiliency.

And on top of the expected technical and project management challenges of producing the StressCam web platform, the team of ECE seniors had to deal with unexpected challenges of a pandemic.

“Before the pandemic, our group would meet in person once a week, and that’s when our best results got created,” Minin said. “COVID-19 also added a difficulty that we couldn’t test our system like we wanted to. Originally, we planned to have a test site set up at the Sandhill Research Station, but obviously that didn’t happen. Instead, Paula set up a test field in her backyard.”

Despite all of the COVID-19-related challenges, Minin found working on an agricultural problem very rewarding.

“I think one of the coolest things about engineering is applying science and technology to an industry that could benefit from it, because otherwise it’s just math on paper,” Minin said. “I really enjoyed working in the agricultural industry, and I think there’s a lot of applications for technology in agricultural industry that haven’t been fully explored yet.”

Ramos-Giraldo agrees about the value of cross-College collaborations.

“It is an amazing effort that we made with the ECE department, and will continue,” Ramos-Giraldo said. “The most important thing is that the students were so enthusiastic about their results. And on the way we learned a lot of things, not only the students, but us as well. It is amazing the results we can produce when we work together.”

The StressCam platform uses IBM’s Internet of Things cloud platform at its core. Additionally, IBM sponsored the senior design project and provided technical mentorship from two researchers based at the IBM Innovation Center, on NC State’s Centennial Campus.

Putting the Stress Cams to Use

Last summer Ramos-Giraldo deployed more than 20 StressCams at the Sandhills Research Station over fields of different soybean varieties. In addition to providing the images used by the Neural Networks class to design the machine learning algorithm, the StressCam images will help NC State and USDA-ARS soybean breeders, including Locke and Tommy Carter, track which of their strains respond best to drought conditions.

This summer, the team has 50 StressCams and plans to deploy them over corn fields. Twelve of the cameras will be deployed in Maryland at the Beltsville Agricultural Research Center, and the rest will be deployed at research stations like Central Crops Research Station in Clayton, where they will take advantage of the new wireless internet infrastructure; Cherry Research Farm, in Goldsboro; and the Caswell Research Farm, in Kinston.

The eventual goal is to roll out the StressCams to the Precision Sustainable Agriculture research network, a network of farms and research stations in 22 states supported in part by a USDA Agriculture and Food Research Initiative grant for the development of resilient agricultural systems.

The StressCams will monitor drought stress in cash crops after planting cover crops, or other alternative cultural practices, said Chris Reberg-Horton, a professor in the Department of Crop and Soil Sciences and co-leader of the research network.

“Our ultimate target with the StressCams is to use them to monitor water status on many, many farms across the U.S.” Reberg-Horton said. “These cameras will help us study the effectiveness of cover crop mulch in capturing extra rain water. They will help us understand how to manage that system to capture the most water and keep it on farmers’ fields.”

Esleyther Henriquez, a master’s student in the Department of Crop and Soil Sciences, will lead an effort to install these cameras on farms across North Carolina and work with the farmers to integrate this new tool into their farm management system.

Funding for developing the StressCam came from USDA National Institute of Food Agriculture.