Overview

This lesson is an introduction to plant breeding and high throughput phenotyping. It outlines the steps and goals of the plant breeding process and identifies areas that can be improved with high throughput phenotyping methods. It covers the steps necessary to collect and process UAV-based high throughput phenotyping data to make reliable plant breeding decisions.

Authors

• Catherine Mick, Department of Agronomy and Horticulture, University of Nebraska-Lincoln
• Don Lee, Department of Agronomy and Horticulture, University of Nebraska-Lincoln
• Leah Sandall, Department of Agronomy and Horticulture, University of Nebraska-Lincoln
• Katherine Frels, Department of Agronomy and Horticulture, University of Nebraska-Lincoln
• Yufeng Ge, Biological Systems Engineering, University of Nebraska-Lincoln

Published 2022

Farming Success in a Changing World

Take a minute to imagine yourself outside on a hot, humid Nebraska summer day. It is probably a matter of minutes before you start sweating through your t-shirt. Luckily for humans, we can go inside and drink an ice-cold glass of water to escape the heat. The day turns to evening and the heat of the day is behind you. It’s the perfect temperature to gather around the bonfire with your friends. Not 10 minutes into your evening you notice tons of mosquitoes won’t leave you alone! Fortunately, your good friend brought some bug spray to protect you!

Just as humans face the elements outside, plants do too. Unlike you and me, plants can’t escape inside or apply a coat of bug spray. There are some agronomic practices like irrigating and applying pesticides, but what about times when this isn’t a possibility? Biotic stressors such as insect pests can become resistant to current methods for control. The planet’s climate change causes increased abiotic stresses such as heat. That’s where the role of plant breeding comes in. Plant breeders work to improve yield by understanding plant genetics and selecting the best genes to maximize crop yield. While pest resistance is one area plant breeders can make improvements, there are many different biotic and abiotic factors that affect yield.

Plant Breeding History

Plant Breeders help farmers succeed in growing crops in a changing world. One of the most important choices a farmer makes every year is what specific genetic type or cultivar they will grow in a particular field on their farm. However, a cultivar that produced a successful crop one year may fail to be successful the next year because of new environmental stresses. The plant breeder only succeeds in helping farmers if they can identify new varieties that have the genetic characteristics to grow well in future and changing environments.

Figure 1 provides a visual testament to the contribution of plant breeders to the farmer’s success. From 1888 to 2018, yield per acre of wheat in the U.S. rose from around 13 bushels to 45 bushels with a surge of improvement from 1940 to 2018. From the 1860s to the 1930s, farmers in the U.S. grew the same varieties of wheat that their relatives grew in Europe or Asia. The old cultivars are sometimes called landraces. Landraces were domesticated and subjected to natural selection in farmer’s fields from the part of the world where the crop species was first cultivated or first adapted by farmers. They saved seed to grow the next year on their farms from what they harvested the previous year. Some farmers traded seed varieties with neighbors or relatives based on the success of the crop. The average yields fluctuated up and down, depending on the stress or lack of stress in that year’s growing season. However, during that time, the average yield trend that we observe in the graph is flat. Farmers were not increasing the yields on their farms. Starting in the 1940’s, the yield trend changed. Plant breeders working for public institutions and seed companies began to work on developing new varieties. These varieties were then chosen by farmers to replace the old landraces and, in combination with improved agronomic management, yields began to increase. The trend in yield improvement continues today.

Plant breeders work closely with farmers to identify the goals they will have in their breeding program. One goal of a wheat breeder, for example, is to create new varieties that can resist the diseases caused by changing populations of pathogens. An average seed company might sell a couple hundred cultivars across the state, nation, or even worldwide. The plant breeder will develop new cultivars. However, these cultivars must meet the farmer’s expectations and grow better on their farm than the cultivars the farmer is currently growing. The breeder, therefore, is motivated to meet the farmer’s standard before releasing one of their new cultivars for seed production. A crop that is widely grown has proven to be successful in being productive on many farmers' fields. Farmers have relied on the cultivars available from this species to grow well. This prior success requires an intense process for developing new, improved cultivars.

Plant breeding is the science of improving crops for human benefit. While the general series of steps remains the same, plant breeding is not a one-size-fits-all pathway. There is a basic template plant breeders follow that they customize for each plant species to optimize their chances of identifying improved cultivars. Therefore, plant breeders will carefully implement practices based on research and theory, and continually search for ways to improve selection methods with new scientifically tested methods.

Plant Breeding Process

The primary responsibility of a plant breeder is to make decisions. They make decisions at every step of the plant breeding process (Fig. 2). There are five main steps in the plant breeding process:

1. Determine the target profiles based on end user needs.
2. Identify germplasm that meets target profiles.
3. Make crosses between plants that are the germplasm or source of genes to make new genetic combinations.
4. Evaluate the genetically diverse plants from these crosses for the traits of interest.
5. Select promising new genotypes to increase the seed yield for final comparison with current cultivars.

Each step requires time, land, money, and labor resources. Every breeding program has limited resources. To be successful the plant breeder must know how to best use the resources available at each step.

Plant breeders are continually considering how they can optimize each step of the breeding process to reduce expenses while increasing efficiency and improving results. One of the factors they must consider when making decisions is how to phenotype the traits of interest. Phenotyping is the process of collecting data on a plant’s observable characteristics to understand how traits are expressed based on the genotype-environment interactions. The data collected from phenotyping is analyzed and used to select desirable genotypes (the genetic makeup of individual plants). A few examples of traits a plant breeder might phenotype are leaf color, plant height, nitrogen use efficiency, or drought resistance.

Often, phenotyping is completed through traditional methods such as manually measuring plant height or weighing bags of seed to measure yield. New technologies such as high throughput phenotyping (HTP) can reduce the labor requirement and collect data more efficiently and effectively. High throughput methods could provide more data collection at important plant growth stages and better inform the plant breeder’s selection decisions.

Plant Breeder Decision Making

Even before making a cross, the first decision a plant breeder must make is determining the trait improvement targets. This can look different depending on whether the breeding program is public (e.g., a university) or private (e.g., agricultural see/biotech company). Determining the target product for release gives the breeder a goal to work towards and they will base all other decisions around that goal.

The genetic improvements needed to achieve the identified target product guide the breeder to make their next decision; which germplasm will they use as parents for their future cultivar? Germplasm is the collection of parent genotypes the breeders can grow to make crosses to start new families of plants. A plant breeder usually prefers internal germplasm that comes from their own program or other programs in the area that is already adapted to their target growing environment. This germplasm is already elite; the genotypes are high yielding and express many of the desirable traits breeders, farmers and consumers want in the plants and the harvested product. Making crosses between improved/elite parents produces a higher percentage of plants that meet farmers’ expectations. The breeder can then determine which of these meets the goals for high yielding breeding lines.

Some breeding goals may motivate the plant breeder to use external germplasm that comes from plant breeding programs outside their region, market class, or growth habit. For example, a maize breeding program may have developed a variety that performs well in drought conditions but is looking to improve the disease resistance against southern rust. They might cooperate with another breeding program that has already made advancements in southern rust disease resistance. This allows breeders to introduce genetic variation to their germplasm pools that is not found within their own (a process called introgression). Using external germplasm makes it more challenging to identify good breeding lines from the crosses because the new parent line may not be adapted to the target environment or end-use market.

Plant breeders also have the option of using germplasm even further outside their region and sometimes country to meet a breeding goal. This is used less often because these potential parents are the landraces, wild accessions, or related species. Landraces may have unique versions of genes (alleles) from traits such as biotic or abiotic stress resistance. However, these landraces are far less productive than elite varieties. A low percentage of the offspring produced using landraces as parents will have productivity that meets farmers standards. But a rare percentage may have unique levels of tolerance to stress which meets the farmer’s goal.

After the breeder decides on the germplasm source, they will make crosses. This is where the breeders take advantage of the genetic variation that resides in different alleles found in the germplasm by selecting pollen from the male anthers and using it to pollinate the female flower. Remember, the goal of plant breeding is to make genetic improvements. When more genetic variation occurs among offspring from a cross, the more potential there is for genetic advancements breeders can accomplish with the breeding process. Sexual crosses allow unique gene combinations from the male and female to generate recombination in offspring which creates a continuum of different genotypes and thus phenotypes that present themselves in the progeny. A plant breeder must have a plan to evaluate this variation.

Evaluating Plant Phenotypes

Breeders must have reliable methods for evaluating plant phenotypes so they can identify the best lines resulting from crosses. Phenotype variation is controlled by both genetic and environmental variation. Some of the traits a breeder must select for in a breeding line are controlled almost entirely by the genotype of the plant. For example, a wheat breeder may have the goal of breeding a new cultivar that has awns (the long extensions of the glume structures that surround the seeds in a head of wheat, Fig. 3A) and the cultivar should also be tall but not too tall which will avoid lodging (falling over before harvest).

Awnless wheat (Fig. 3A left) is the homozygous recessive genotype aa and awned (Fig. 3A right) is homozygous dominant AA or heterozygous Aa. Variation in the growing environment will not influence the development of the awned trait. In contrast, variation in height of the wheat plant is controlled by many gene pairs but will also respond to water availability. Plant breeders will measure the height of the same genotype in multiple environments to have confidence they can predict the plant height in a farmer’s field (Fig. 3B). It is crucial for the plant breeder to know to what extent environmental variation will affect the traits they need to evaluate to meet their new cultivar goals. These are the traits that will demand more of the time and land use resources of the plant breeder’s program. Therefore, plant breeders are continually searching for proven technologies that have the potential to more efficiently use resources to evaluate phenotypes that are affected by both genetics and the environment.

Can New Technology Help Plant Breeding Decision Making?

Plant breeding has historically been a highly manual process. Breeders and their teams rely on their hands, eyes, and physical effort to collect phenotype data for evaluating genotypes in their programs. The breeding team walks the field, measuring traits from plot to plot. This is a time-consuming and labor-intensive process. Phenotyping for some traits is a major bottleneck in maximizing the efficiency of the plant breeding process. Advances in sensor technologies and UAVs offer the potential to transform this process by replacing much of the manual labor with automated, high-throughput phenotyping (HTP). These advanced sensor technologies can collect millions of measurements in a short time, dramatically reducing the time, effort, and cost required. The more phenotypes that are evaluated for each genotype, the easier it is to find the ‘one in a million’ cultivar which will eventually reach the producer. Plant breeding is a numbers game: the larger the volume of phenotyping we can do, the greater the probability we have for genetic gain. Capitalizing on high throughput phenotyping technology could improve the evaluation step in the breeding process and accelerate breeding accomplishment.

High Throughput Phenotyping Technology

One way to classify HTP methods is to separate them into four categories: Field-based phenotyping platforms (FBPP), cable-suspended, satellite imaging technology, and aerial vehicles. FBPP and cable-suspended HTP devices are limited by their inability to conduct large-scale phenotyping and inflexible maneuverability while satellite imaging technology is limited by their spatial and temporal resolution (Bai et al., 2016; Bai et al., 2019). Aerial vehicles includes both manned and unmanned aerial vehicles (UAV, aka drones). Of the two, UAVs are becoming a more enticing option for phenotypic data collection due to their high spatial and temporal resolution and flexibility in spectral resolution and maneuverability (Mick, 2023). For additional examples of some of these platforms you can find information in the Song et al. (2021) article “High-throughput phenotyping: Breaking through the bottleneck in future crop breeding”. These platforms are used to carry different sensors that can measure phenotypes. Sensors can be mechanical, chemical, or optical. Mechanical and chemical sensors measure plant morphological and chemical traits using a physical sample of the plants (Burrell et al., 2017; Forristal & Keppel, 2001). Optical sensors can measure these characteristics in a non-destructive manner making them a more ideal option for plant phenotyping.

Like our eyes, optical sensors are instruments designed to detect specific wavelengths of light in the electromagnetic spectrum. The electromagnetic spectrum is the entire range of electromagnetic waves organized by wavelength or frequency (Fig. 4).

Our eyes and brain are biologically designed to detect and recognize the wavelengths in what is called the visible light region of the electromagnetic spectrum. HTP sensors are engineered to measure the level of reflectance and absorption of light in the entire electromagnetic spectrum. Different sensors are sensitive to a different range of wavelengths of light. Table 1 provides a general range of wavelengths measured by each sensor type, but the specific wavelengths captured vary depending on the sensor brand used.

This range of wavelength detection sensitivities provide plant biologists and plant breeders combinations of sensors to test which can measure differences in how plants interact with (absorb / reflect) light. The question is ‘do these light reaction differences relate to the traits the plant breeder needs to evaluate?

Plant Structure and Light Interactions

One of the primary functions that helps a plant grow is photosynthesis. During photosynthesis a plant absorbs light energy and converts it to chemical energy. Light absorption happens in a molecule called chlorophyll. Figure 5 shows the percentage of visible light that is absorbed by chlorophyll (two pigment molecules) that would be found in a healthy plant leaf that is supporting an actively growing plant and by β-carotene, the pigment made in bright orange pumpkins, fruits, or carrot roots. Chlorophyll a (depicted by the solid line) and chlorophyll b (dashed line), absorb primarily blue and red wavelengths. The wavelengths of light that are not absorbed for photosynthesis are reflected. Plants reflect more green light than red or blue, which gives them their green appearance. What we cannot detect with our eyes is that healthy plants reflect a significantly higher proportion of NIR wavelengths compared to visible green light. When a plant becomes unhealthy, it reflects less NIR wavelengths and more blue and red wavelengths.

We can use Figure 6 to compare the information generated from a sensor to information we are accustomed to obtaining from our eyes and brain. Assume each line is the reflectance of light data obtained by a sensor collecting data from three different fields.

See if you can answer this question: Which field in Figure 7 is the solid line, which field is the dashed line and which field is the dotted line in Figure 6?

The visual data from Figure 7 we routinely interpret tells us that field A is green and healthy, field B is brown soil and field C is yellow and less healthy. The first way we connect the data from Figure 7 and Figure 6 is to recognize that we should be looking at the visible light wavelengths (450-650 nm) in addition to the NIR wavelengths (700-900 nm). Figure 6 shows us the spectral reflectance for the three different fields. The dotted line is uniformly reflecting light at all those wavelengths increasing from the visible to the NIR region. This spectral signature is indicative of soil and is useful when distinguishing the difference between soil and vegetative spectral signatures. Thus, the dotted line would be the soil (Area B).

The solid line on the graph has two peaks – one in the green band and one in the NIR band. This indicates the field providing data for the solid line is reflecting light in mostly the green and NIR wavelengths. Using this information and your understanding of plant light reflectance for healthy versus unhealthy vegetation, you can match the solid and dashed lines with the plants reflecting different amounts of visible and NIR wavelengths. This activity illustrates how sensors provide data that our eyes and brains cannot. We should be familiar with the choices of sensors available to collect this reflectance data.

High Throughput Phenotyping Sensors

Red-Green-Blue

RGB sensors detect wavelengths of light in the red-green-blue spectrum to create a picture much like how you and I see the world. An RGB sensor mounted on a UAV captures numerous overlapping images throughout its flight, enabling researchers to generate an image of the entire field during the data processing stage. The final product will look similar to Figure 8. The data captured by an RGB sensor can also be used to calculate vegetation indices. Vegetation indices can be used to understand plant health and biomass.

Multispectral

Multispectral sensors are designed to collect data outside the visible light spectrum. This includes wavelengths in the UV and NIR ranges. The advantage to this is that a multispectral sensor can provide a picture of the entire field (like an RGB sensor) but can also be used to calculate more vegetation indices than RGB sensors. The detection of these additional wavelengths provides a more sophisticated understanding of plant health and biomass.

Hyperspectral

Hyperspectral sensors capture a continuous range of wavelengths across the electromagnetic spectrum (Fig. 5). These sensors measure hundreds of wavelengths of light at once including those undetectable by the human eye. This information is useful when the specific wavelengths of light needed to measure a trait are unknown, making hyperspectral sensors good for exploratory research.

LiDAR

LiDAR stands for Light Detection and Ranging. LiDAR is known as an active sensor meaning it sends out a pulse of light which gets reflected back to the sensor, as compared to most sensors used in HTP that sense the light reflected from the sun. LiDAR measures how much time passes between emitting the light and receiving it back. This measure of time is then used to calculate the distance between the sensor and the point the light reflects off. LiDAR sensors generate data points as they move across a field or area. Each point has an x, y, and z coordinate to give it a location in space. Through data analysis we can create 3-dimensional data and measure a physical feature such as plant height. This type of data collection typically requires high-resolution global positioning system (GPS) data or 3D ground control points (GCP), or both, for reconstruction.

Thermal

Thermal sensors are sensitive to wavelengths in the infrared region. Because infrared wavelengths are undetectable to the human eye, thermal sensors can identify plant stress before humans can. This makes them useful when it comes to water stress and disease detection.

Selecting a Sensor for a Specific Trait

One of the benefits of using UAVs in plant breeding is the ability to mix and match sensors to fit your needs. For example, RGB and LiDAR sensors can be useful for measuring physical traits like plant size, canopy cover, or height. If you want to look deeper into the plant to detect disease or pest infestations, thermal sensors would be more useful. If you are conducting exploratory research and not sure what sensor to use, the hyperspectral sensor will be useful because it can measure every wavelength allowing you to determine which wavelengths correlate with your trait of interest. When selecting a sensor, it is important to consider the traits of interest, technical expertise required, and budget.

While all sensors are being used and studied in agriculture, we will focus the rest of this lesson on implementing HTP into a breeding program to measure biomass. Biomass can be measured by a breeder with ‘hands on’ methods. Biomass, however, has a strong correlation with a sensor data generated value called the Normalized Difference Vegetation Index (NDVI). NDVI is one of the most commonly used vegetation indices. Vegetation indices can provide us with valuable information about the growth and stress crops are experiencing. They can also be used to differentiate between plant and non-plant material.

Two sensors a plant breeder would find useful for measuring biomass using vegetation indices are RGB sensors and Multispectral sensors. RGB sensors can be used to measure biomass but multispectral sensors provide a more accurate measure of biomass due to their sensitivity of the NIR wavelengths that can be used to calculate vegetation indices.

UAV Flight

The available sensors can collect data that provide valuable insights for plant breeders when attached to a UAV. The sensor and UAV together form an unmanned aerial system (UAS). The key to collecting a complete data set of all the plant breeder’s plots at a specific location is to plan and execute a good flight. The flight plan and organizing the flight operation in the field require attention to details.

After the UAV pilot arrives in the field, they will set up ground-control points (GCPs, Fig. 9). GCPs are points on the ground that have a set of known GPS coordinates. GCPs are essentially large targets for drones to use as geo-reference points. Having these aids in accuracy during post-processing of the UAS data. To set up GCPs, the researcher will evenly distribute 4-8 targets around the perimeter of the field and one in the center of the field. It is important that GCP targets are placed in an unobstructed location, easily visible by the drone. Using a GPS device, the researcher will store points for each GCP target. GCPs are not always permanent objects, so it is important to mark the location of the target so it can be placed in the same position every time a flight is conducted.

The next step is to prepare the drone for flight. This may look different depending on the type of drone being flown. Smaller, more commercially used drones typically require less setup and may already have sensors attached. Large drones often need some assembly. This may include unfolding/attaching propellors, attaching battery packs, inserting the data chip, and attaching the appropriate sensors. Initially some calibration steps may be necessary as well to ensure the sensor is working correctly and accurately.

Following the calibration is the actual flight and data collection. When the drone is flying, it is switched over from manual control to ‘autopilot’ following a predetermined path programmed by the pilot.

Creating a flight plan can be done prior to leaving for a field or after arrival. While different drones and software have their own nuances, the major steps to creating a flight plan are the same. The first step is to create a project. Wherever you are located at the time of creating the project is where the software is going to initially place your project. If the flight plan is created away from the field that has the breeders’ plots, the project can be moved around to find the right location. Once you determine that, you can edit the boundaries for your project to ensure the flight captures your desired area.

After you have set your boundaries, you are going to change some advanced settings to fit your needs. Some notable settings include height, overlap, and gimbal angle. The height is important to your mission because it determines how high the drone will be flying over the crop. No matter how many times you fly the mission to collect data, the height should stay the same. Because of this, you want to make sure you choose a height that is taller than the crop will be later in the season. The height a drone flies determines the number of pictures a sensor is going to take and the resolution of the pictures. The lower the height, the more pictures and higher resolution. The higher the height, the less pictures and lower resolution (Fig. 11). Plant breeders in Nebraska work with maize, soybeans, and wheat. To make sure we get a quality image, it is common to fly around 25-30 meters. While this does increase image resolution, it also increases the time it will take a drone to complete a mission and subsequently the number of batteries needed to complete a mission.

There are two settings to adjust when changing the overlap on a mission. These are the forward and side overlap (Fig. 12). As you can imagine, this determines how much the images overlap, an important aspect when it comes to stitching the photos together during data analysis. The overlap allows the images to be merged more easily. While this is at the discretion of the pilot/researcher, it is common practice to set the overlap between 70%-80%. Gimbal angle is the angle at which the camera or sensor is turned. When conducting 2D flights over crops, it is important to set the gimbal angle at –90 so the sensor is looking straight down at the crops. Gimbal angles other than 90 degrees can be used when objectives are more aligned with 3D reconstruction data.

Other settings to choose are speed, shooting mode, photo ratio, and white balance. Each of the settings affects the quality of data you will collect so it is important to determine and set these appropriate to your project.

At this point, you should double check your flight plan and drone to ensure everything is correct. If you have determined it is safe to fly, the mission is started at which point the drone is switched over from manual control to ‘autopilot’ following the predetermined path the pilot programmed prior to traveling to the field site.

During the flight, any data collected is saved on a data chip for easy extraction. Once the flight is complete, the drone returns to the landing pad, is packed up, and the data is stored to be processed back at the office.

Attaching Sensors and Autopilot - video showing how to attach cameras to larger drones and what the tablet looks like while the drone is collecting data (no audio on the video).

Drone Setup - video showing how to set up a larger drone (no audio on the video).

 

Ground-Truthing

A plant breeder must impose selection based on a reliable evaluation of hundreds of different genotypes to identify breeding lines that are an improvement over what a farmer currently grows. Thus, HTP must go through rigorous testing if it will replace or add to the breeder’s current evaluation plan. One part of this research process is called ground-truthing (also sometimes called ground referencing). Ground-truthing is exactly what it sounds like: the data collected manually at the field location is assumed to be a true measurement of the plant’s phenotype. The data collected by the UAS can be compared with the data collected by the normal field methods. Ground-truthing then is the process of determining if a particular, repeatable method of analyzing the sensor data ‘lines up’ with the differences measured with hands on methods. Ground-truthing is how breeders know UAS are accurate, because this data collection method is used as the standard to which data collected from high throughput phenotyping technologies is compared using statistical analysis. Collecting ground-truth data can be time consuming and physically demanding. Fortunately, as advances are made in high throughput phenotyping technologies, we can phase out the need for ground-truthing.

In current programs, ground-truthing is used to measure traits which plant breeders categorize into one of two groups: basic traits and leaf-level traits. Basic traits are those we can see with our eyes which allow us to perform a visual or manual measurement to collect data. Examples of basic traits include flowering time, emergence, and plant weights.

One basic trait of interest to breeders is height. Breeders are interested in plant height because it is correlated with plant health. Plant height is also being used to predict final yield and biomass. Plant height is one of the simplest traits to gather ground truth data on because of how it is collected. To gather height data, researchers will go out into the field with a meter stick and clipboard. In the case of height, researchers had to measure the height multiple times across many environments to get an average (Fig. 13). Based on these measurements they then selected the best breeding lines to move forward in the breeding process.

Leaf level traits require more effort to score because these traits are not reliably detectable to the human eye. Leaf level traits such as chlorophyll concentration and light reflectance can be measured with hand-held sensors. Breeders are interested in chlorophyll concentration and leaf-level light reflectance because they are strong indicators of plant health. The leaf area index (LAI) can also be measured by sensors. LAI is the ratio of leaf area per unit of ground. Healthy plants can grow at different rates so using leaf level measurements to quantify the amount of vegetation in a specific area of the plot is an important trait.

Measuring leaf area index can be done either directly or indirectly. Direct measurements are generally more accurate, but it comes at the price of being highly destructive. One way direct measurements are done is by harvesting all the leaves in a defined area and measuring the area of each leaf to determine the total. This is classified as destructive sampling because the plants are removed from the plot to collect the data. Another method is doing litterfall collection. This is done by collecting fallen leaves in traps and calculating leaf area index by multiplying dry mass by the specific leaf area. This process is done over a period of time to calculate leaf area index. There are two major downfalls to the second method. The first is that it is labor intensive. The litter fall traps must be frequently emptied to avoid decomposition and biomass loss. The second is that the specific leaf area used in calculations is only an estimate, so there may be some inaccuracy.

Indirect measurements have a slight edge over direct measurement methods because they are nondestructive and not as time-consuming. Indirect measurements are possible because recent research has developed ways to correlate these with direct measurements. For biomass specifically, research has shown that leaf area index can be a good indicator of biomass. To measure leaf area index, a plant canopy analyzer called a ceptometer can be used (Fig. 14). A ceptometer measures photosynthetically active radiation (PAR). To measure leaf area index, a thin, long sensor called a probe is placed underneath the plant canopy to measure the amount of PAR that travels through the canopy. Simultaneously an external sensor is held above the plant canopy to measure the amount of PAR coming in (Fig. 15). Both measurements are used to calculate the amount of PAR that is blocked by the canopy which correlates to the amount of leaf cover and thus leaf area index.

Data Analysis

Flying a field and collecting data is only one small part in implementing HTP into a breeding program. There are many environmental factors that can affect the quality of images captured by sensors: row angle, sun angle, time of day, consistency over time, etc. Due to these factors, data collected will need to go through a series of data processing steps to remove any inconsistencies before it is useful to a breeder.

One type of information to extract from sensor data is spectral information. Spectral information is the wavelengths of light measured and the corresponding data they provide. For example, we are gathering information about a cat. We want to identify the animal as a cat, perhaps as a specific breed of cat, and eventually identify it as a specific cat. The spectral information would tell us things like the color of the eyes or color of the hair. Arrangement of color on the body. Sensors can also extract structural information to provide size, proportions of the body, legs and head and hair length. Plant breeders will focus on the spectral information from sensors and supplement it with structural information to gather the best data that reveals the phenotype of the plant at the time of the measurement. Structural information is often used to supplement the vegetation indices. Alone, each type of information is useful, but combined, can put together a full picture of the health status of a plant.

There are three main steps that make up processing of the raw data that the UAV collects from sensors in a specific flight: Stitch the images, create a shapefile, extract indices.

Data Processing

Stitch Images

When you fly over a crop collecting UAS imagery, you can collect hundreds of photos that cover the entire field. These images are not useful on their own, so part of data processing is to stitch all these images together. There is software readily available and user-friendly that can do this. One common software is Pix4D which can be accessed by purchasing a subscription. To create a mosaic from your images, you just need to upload them to the software and change the settings to your preference. This is where the GCP coordinates are used to correct any distortions. The challenge with stitching images is the processing time and computing power needed to run the software. In general, as the field size you are imaging increases, your project size will increase and require more RAM, CPU, storage space and processing time.

Every one of these images you create is actually a matrix containing information. If you zoom in on any point within the matrix, the value of that pixel is an element in an array. The final processed image from an RGB sensor is three, two-dimensional arrays laid on each other to create a matrix (Fig. 17). Similarly, an image from the example multispectral sensor is five, two-dimensional arrays laid on each other to create a matrix. Each array containing the values for the NIR and red-edge reflectance in addition to the RGB reflectance values. This is how the images can be used to extract objective information rather than subjective information.

Create Shapefile

The RGB file that is created from stitching all the continuous images together can be used to create a shapefile using a polygon grid. A polygon grid is just a grid that divides a file into cells based on the coordinate system. This polygon grid will allow us to systematically ‘clip’ the NDVI raster file to extract NDVI for each individual plot. There are many accessible software programs for users interested in creating a shapefile. Two commonly used programs to do this are ArcGIS and QGIS. The main difference between the two is that ArcGIS is proprietary software whereas QGIS is an open-source software. Every software has benefits and detriments, so it is up to the user to determine which is the best fit for their analysis.

Extract Indices

One of the biggest areas of interest when using a multispectral sensor is the ability to calculate vegetation indices. This step is the most technical and requires a basic understanding of programming languages to extract data from the images. We can calculate the vegetation indices for the entire field using software called ArcGIS. There is a raster calculator that can be used to process the initial image and calculate vegetation indices. However, to extract more advanced vegetation indices will require the use of a programming language like Python.

The processed image will be saved as a raster file which allows us to extract information. The way a raster file works is that it is a matrix of pixels. If you zoom in enough on a small section of a selected plot, you will end up with a bunch of pixels. Every pixel is going to have its own value. To extract the vegetation index for a specific plot, requires the use of a programming language like Python. We can write a code that uses the shapefile with a polygon grid created earlier to clip the vegetation index raster file into individual plots. The average of the vegetation index is then calculated using every pixel within the area of a specific plot.

Some common indices calculated from multispectral imagery include the Normalized Difference Vegetative Index (NDVI), Normalized Difference Red Edge (NDRE), and the Green Normalized Difference Vegetative Index (GNDVI). Some common indices calculated from RGB imagery include the Green Leaf Index (GLI) and the Visual Atmospheric resistance index (VARI).

[latex]NDVI = \frac{NIR - Red}{NIR + Red}[/latex]

[latex]NDRE = \frac{NIR - RedEdge}{NIR + RedEdge}[/latex]

[latex]GNDVI = \frac{NIR - Green}{NIR + Green}[/latex]

[latex]GLI = \frac{(2 \times Green) - Red - Blue}{(2 \times Green) + Red + Blue}[/latex]

[latex]VARI = \frac{Green - Red}{Green + Red - Blue}[/latex]

All of these indices will fall between a range of -1 to +1. Think back to when we discussed which wavelengths reflect off plants. In the visible spectrum, green wavelengths are going to reflect off the plant, causing the vegetation to look green. Red and blue wavelengths of light are used to drive photosynthesis. This provides plants with food to grow, and aids in chlorophyll production. NIR bands are also reflected by the chlorophyll in the plant. When calculating the vegetation indices for a healthy plant, there is going to be high chlorophyll production resulting in a higher amount of light reflectance. This provides vegetative index close to +1. If a plant is not healthy or experiencing stress, there is going to be less chlorophyll production which in turn lowers the amount of light reflectance. This would give a vegetative index close to 0. Extracting the vegetation index for individual plots provides the plant breeder with the information they need to make decisions.

Making Decisions

The plant breeder can examine the data they have extracted for each plot and decide which breeding lines will be moving forward in the trial. Suppose Table 2 contains four varieties you are working with. As a breeder, you are concerned with increasing the amount of biomass produced. Because NDVI is positively correlated with biomass, as a breeder you can select the varieties that have the highest NDVI rather than harvesting each plot for destructive biomass measuring.

Table 2. Four plant varieties ranked from best (1) to worst (4) based on NDVI. Each variety has a unique Plot ID signifying planting location in field.

Plot ID	Variety	NDVI	Ranking
837	N5536	0.666813	1
278	N7236	0.354654	3
094	N2938	0.234223	4
290	N8478	0.564904	2

Figure 18 is a more visual way to understand the spectrum of values you could get when using vegetation indices to assist with selection. Imagine you are a plant breeder trying to select which of these four plants to advance. It may be obvious to not advance option A and B, but what about C and D? Using just your eyes, you may have a hard time distinguishing the difference in vegetative health or density. Calculating vegetation indices allows plant breeders to make selections based on objective measurements. This allows them to advance only the best varieties in an efficient manner.

Limitations to High Throughput Phenotyping

HTP offers many advantages to the plant breeding industry such as reducing the time required for some data collection like plant height. It is also important to discuss some limitations of the technology. A few considerations are:

• Data storage – HTP can collect immense amounts of data, so you need the capacity to store all that data. To give you an idea, saving multispectral images collected by a UAS flown at an altitude of 25 meters over a field that is ~6 acres will require about 15 gigabytes of storage. This is just one example, but as a general rule the size of the field, altitude of flight, overlap, and photo intervals are all going to affect the storage space needed. You will also need to have a proper amount of storage for the processed data.
• Processing Time – This technology increases the volume of data while decreasing the amount of time required for manual data collection. However, there is additional time needed to process the data after collection. For example, to manually measure plant height, LAI, and destructive biomass in 30 research plots will take a team of 5 individuals about 4 hours. To capture this data with a UAS will take one person about 30-40 minutes. The images collected by the UAS will then take roughly 5-6 hours to process.
• Price – Depending on the drone and sensors used, these UAS can come with a big price tag. The DJI Phantom 4 (Fig. 10) is purchased with the sensor included and is on the cheaper end when it comes to UAS. In comparison, the DJI Matrice 300 (Fig. 10) without the sensors included costs approximately six times as much as the DJI Phantom 4. In addition, purchasing the most commonly used RGB and multispectral sensors to attach to the DJI Matrice 300 UAV will double the price. There are also other costs associated with accessories like batteries and purchasing software to process the data.
• Software – This technology is still being developed, which presents a few issues. During drone setup the software can glitch causing startup to take longer than just ‘pushing a button and turning it on’. This can be frustrating for researchers who have a limited window to collect data due to the weather and plant growth stages.
• Indirect measurements – We cannot directly measure every trait of interest, so we must use indirect measurements of traits that correlate with those we are interested in. For example, we cannot distinguish diseases in crops using UAS. Instead, we can use sensors to determine plant health based on the calculated NDVI. Plant health can be correlated with the amount of disease present in a crop. However, plant health can be affected by many factors like drought conditions and nutrient deficiencies, so it is important to still be present in the field to make visual observations.

Despite these concerns, researchers believe UAS will have a beneficial impact in the agriculture community. Scientists are continuing to conduct research to minimize these issues and improve the technology.

Use of High Throughput Phenotyping Today

This lesson presented just one of many applications HTP devices are used for today. The implementation of high throughput phenotyping varies widely across breeding programs. Some programs have very established high throughput phenotyping methods, so they are used as the main method of data collection without the use of ground truthing. Other programs like the University of Nebraska-Lincoln Small Grains Breeding program have not yet fully integrated high throughput phenotyping into breeding pipelines and are conducting research to prove we can trust these methods of data collection. Overall, high throughput phenotyping is a tool that is both efficient and objective which can revolutionize the volume of plant breeding we can accomplish.

Acknowledgements

Development of this lesson was supported in part by the U.S. Department of Agriculture, National Institute of Food and Agriculture project 2020-68013-32371. Any opinions, findings, conclusions or recommendations expressed in this publication are those of the author(s) and do not necessarily reflect the view of the U.S. Department of Agriculture.

References

Bai, G., Ge, Y., Hussain, W., Baenziger, P.S., & Graef, G. (2016). A multi-sensor system for high throughput field phenotyping in soybean and wheat breeding. Computers and Electronics in Agriculture, 128, 181-192.

Bai, G., Ge, Y., Scoby, D., Leavitt, B., Stoerger, V., Kirchgessner, N., Irmak, S., Graef, G., Schnable, J., & Awada, T. (2019). NU-Spidercam: A large-scale, cable-driven, integrated sensing and robotic system for advanced phenotyping, remote sensing, and agronomic research. Computers and Electronics in Agriculture, 160, 71-81.

Burrell, T., Fozard, S., Holroyd, G.H., French, A.P., Pound, M.P., Bigley, C.J., Taylor, C.J., & Forde, B.G. (2017). The Microphenotron: a robotic miniaturized plant phenotyping platform with diverse applications in chemical biology. Plant Methods, 13, 10.

Forristal, P.D. & Keppel, D. (2001). The Application of Harvester -Mounted Forage Yield Sensing Devices. End of Project Reports, Teagasc.

Mick, C. (2023). High Throughput Phenotyping: Field Based Triticale Breeding and Educational Resource Impact [Masters thesis, University of Nebraska-Lincoln].

Song, P., Wang, J., Guo, X., Yang, W., & Zhao, C. (2021). High-throughput phenotyping: Breaking through the bottleneck in future crop breeding. The Crop Journal, 9(3), 633-645.