Introduction

Tulip and other bulbous ornamental crops are often infected by viruses. One of the most important viruses in tulips is the potyvirus TBV (tulip breaking virus). Symptoms of the virus can manifest themselves in different ways, including striping of the leaves and abnormal flowers (Dekker et al. 1993). The presence of viruses causes a reduction in the quantity and quality of the product, which leads to sales and export restrictions. In the Netherlands the tulip area covers about 11 000 ha, which is about 50% of the total area of flower bulb cultivation. In 2008, about 2.3% of the tulips were infected with TBV. Current methods to keep the disease under control are twofold. Firstly, plants are sprayed with chemicals to control aphids, which spread the virus (Asjes 1975; Asjes and Blom-Barnhoorn 2001). Secondly, crop experts go through the field and remove plants with visual symptoms. In the Netherlands the annual cost associated with tulip breaking virus in flower bulb culture is estimated at 9 M€ (van der Vlugt 2006).

A major problem with visual assessment of infected plants is the difficulty of observing symptoms, which requires an expert eye. The visibility of the symptoms is also largely influenced by the tulip cultivar and weather conditions, and so can often only be seen during a limited period of the growing season. This requires a large number of trained personnel, which can be difficult to find for a short period of time.

The control of plant viral diseases in ornamental crops, such as tulips, by plant protection agents is becoming increasingly restricted because of environmental concerns. Integrated crop protection management systems are still in development. Certain chemicals are restricted, therefore, chemical-free methods, such as resistant varieties, are needed to control the incidence of virus-infected tulips. Although some tulip varieties exist that are less susceptible to infection (Romanow et al. 1990), viral infections are still an important problem for tulip bulb growers.

To reduce the amount of chemicals needed to control aphids and the considerable pressure on labor, there is an urgent need for a rapid and objective method of screening to identify and remove infected bulbs. Molecular techniques provide an interesting option, as they would enable accurate testing of pathogen species and even a strain in a specific way. In addition, it would allow testing of bulbs during the storage period, which would provide a larger time frame compared to testing during growth in the field. However, the high costs of testing per bulb, the complexity of the automation process to obtain a sample in the storage room and possible damage to the bulbs when taking a mechanical sample from them hamper the practical application of these techniques. As an alternative, proximal (close-range) optical sensing techniques can help in the control of the disease by automatically identifying infected tulips. Imaging techniques have shown their potential for applications such as yield mapping (van der Heijden et al. 2007), early stress or disease detection (Meroni et al. 2008), weed management (Yang et al. 2003) and fertilizer applications in precision farming (Jørgensen and Jørgensen 2007). However, with such techniques potential sources of error, such as operational characteristics of the sensor, seasonal variation, water status, irradiance, growth stage and plant genotype have to be taken into consideration (Samborski et al. 2009).

Hyperspectral and multispectral imaging have proved to be promising techniques for detecting bacterial and viral diseases in crops. Optical properties have been studied in relation to the severity of bacterial leaf blight in rice using canopy hyperspectral reflectance (Yang 2010). Comparing spatial and temporal patterns in crop spectra can provide signatures for characterizing and estimating fungal disease severity in a spring wheat crop (Muhammed 2005). The small computational load of the proposed approach makes it suitable for real-time on-vehicle applications. A hyperspectral computer vision system (Gómez-Sanchis et al. 2008) is capable of detecting damage caused by Penicillium digitatum in mandarins using a reduced set of optimally selected bands. Fluorescence imaging provides a non-invasive technique to screen the photosynthetic processes of plants (Krause and Weis 1991). Biotic stress in leaves exposed to a pathogen showed heterogeneous responses when subjected to fluorescence imaging (Chaerle et al. 2007).

In this study, experiments were carried out to test the detection performance of several optical sensors for viral symptoms in tulips under controlled laboratory conditions. The methods of testing were restricted to those that have the potential to be scaled up to practical applications in the field. Thus, highly sensitive and or expensive methods such as magnetic resonance imaging (MRI), X-ray or Raman-imaging are not considered. The long term aim is to develop a robot system incorporating the most promising vision technique obtained from this study that can automatically detect and remove diseased plants in the field. The application of this robot system will reduce the amount of insecticides and the considerable pressure on labor for selecting diseased plants by the crop expert.

Materials and methods

Experimental setup

Three tulip varieties (Barcelona, Monte Carlo, Yokohama) that showed a range of symptoms from TBV were selected for the experiment. To have sufficient infected bulbs for testing, sets of bulbs were selected that had high rates of TBV infection. These bulbs had been assessed in the former breeding season by ELISA, an enzyme-linked immunosorbent assay (Derks et al. 1982). The established infection rates for the bulb sets were 14% in Yokohama, 16% in Barcelona and 31% in Monte Carlo. For every cultivar 400 to 800 bulbs were planted in small plastic baskets and buried in the field. Early in the growing season (March 2008) individual plants were assessed visually by trained experts and marked when TBV symptoms were present. Afterwards, leaves of about 100 visually healthy and 100 visually infected plants were selected and measured using four different optical sensors.

The ELISA analysis, using TBV-specific antisera in a validated protocol (Derks et al. 1982), was carried out on the same leaves as those measured with the optical sensors. These ELISA measurements were used as the reference by which to compare the human visual assessment and the optical sensor techniques.

Both the visual assessment and ELISA results are denoted as scores, whereas the analysis of the data of the optical techniques is denoted as classification results.

Optical techniques

Four different optical sensor techniques were tested in this study:

  • an RGB color camera,

  • a spectrophotometer with a spectral range from 350 to 2500 nm,

  • a spectral camera with a range from 400 to 900 nm,

  • a chlorophyll fluorescence imaging system (Jalink et al. 2004; Polder et al. 2007), which measures the photosynthetic activity.

RGB color camera

A digital picture of each leaf examined was taken under controlled lighting conditions. The camera used was a Nikon D70 with a NIKON 18–70 mm zoom lens in a closed cabinet equipped with high frequency fluorescent illumination (Osram Biolux daylight tubes). To correct for possible changes in the illumination, a Macbeth color chart was included in every image. This made it possible to check whether there were color changes and to correct for them if necessary. To identify each leaf, each image was coded by a 2D QR-barcode label in the image. Figure 1 shows a typical image. After recording, the images were segmented and spatial information was extracted by calculating several shape properties for each leaf. In addition to simple features such as the dimensions and intensity of the object, other more complicated ones were derived from the image data. The moments of inertia were calculated using the eigenvalues of the object’s second-order moments tensor (Jahne 1997). These features give information about the extension of the leaf in the main direction (length) and the perpendicular direction (width). The bending energy is a measure of the mean curvature along the contour and is directly proportional to the bending energy of a deformed circular rod (Young et al. 1974). It is used here as a measure of the roughness of the contour of the leaf. The Podczeck shape factors use the deviations of the two-dimensional particle outline from the standard outline of a circle, triangle and square, and consider the particle elongation and number of characteristic corners of the apparent shape (Podczeck 1997). Table 1 gives a list of all shape properties used in this experiment.

Fig. 1
figure 1

A typical RGB image of a tulip leaf with Macbeth color standard and barcode

Table 1 The calculated shape properties of leaves recorded with the RGB color camera

Infected plants often have a red or purple spot pattern on the leaves. These spots were quantified using color segmentation. Subsequently, the total area and perimeter of the spots, and the number of separated spots were calculated (Fig. 2).

Fig. 2
figure 2

a Image of infected tulip leaf, b segmented image, c pattern of color, d contour pixels of color pattern and e number of spots

Spectrophotometer

The spectrophotometer used was a FieldSpec Pro FR spectroradiometer from Analytical Spectral Devices (ASD). The total range in wavelength was from 350 to 2500 nm, with a resolution of 3 nm in the visible range and 10 nm in the short-wave infrared. It provides detailed information on the spectral reflectance of the measured surface (Clevers et al. 2008). A leaf clip was used to measure the reflectance spectrum of a circular part of the leaf with a diameter of 2 cm in a standardized way (ASDi 2010). Two non-overlapping spots were measured on each leaf, resulting in two reflectance spectra per leaf (Fig. 3).

Fig. 3
figure 3

Position of the circular measurement spots of the spectrophotometer. The diameter of the spot is 20 mm

The spectrophotometer consists of three different sensors, therefore, a small mismatch between adjacent spectral regions may occur. Each spectrum is corrected for this mismatch before further processing using standard protocols. The output from the spectrophotometer is 2151 data points per spectrum. This is more information than is needed for classification and also more than the physical resolution of the sensors. Therefore, the data were reduced to 70 points with Savitzky–Golay smoothing (Savitzky and Golay 1964) followed by down-sampling by taking each 30th data point from the smoothed spectrum.

The analysis of the spectrophotometer data was required to answer two questions: the feasibility of distinguishing between healthy and diseased plants using only spectral information and the possibility of using the data, especially in the near infrared region, for assessing disease severity. The near infrared region is also used to determine whether more expensive sensors with sensitivity beyond 900 nm are needed. A different way of quantifying the importance of each spectral region for the classification between healthy and virus-infected plants was obtained by studying the classification error for small parts of the complete spectrum. Small sub-spectra of 100 nm interval, starting at every 10 nm increment were used. This results in a 90 nm overlap between each consecutive sub-spectrum. The magnitude of the classification error of these small spectra indicates the importance of the spectral region studied.

From the literature, a large number of vegetation indices are known to discriminate between different plant properties. The following indices were examined to determine whether they can be used to identify TBV diseased plants:

  • Photochemical reflectance index (PRI) = \( \left[ {{\frac{{R531 - R570}}{{{{R531}} + {{R570}}}}}} \right] \) (Gamon et al. 1990),

  • Chlorophyll index (green) (Chlgreen) = \( \left[ {\frac{R780}{R550} - 1} \right] \)(Gitelson et al. 2006),

  • Chlorophyll index (red-edge) (Chlred) =\( \left[ {\frac{R780}{R710} - 1} \right] \)(Gitelson et al. 2006),

  • Anthocyanin reflectance index (ARI) = \( \left[ {\frac{1}{R510} - \frac{1}{R700}} \right] \) (Gitelson et al. 2001),

  • Modified anthocyanin reflectance index (mARI) = \( R780\left[ {\frac{1}{R550} - \frac{1}{R700}} \right] \) (Gitelson et al. 2006),

  • Red:Green Ratio (RGR) = \( \left[ {\frac{R660}{R550}} \right] \) (Gamon and Surfus 1999; Sims and Gamon 2002),

where is the reflectance at λ nm.

The PRI index measures photosynthetic activity, which is expected to be affected by diseases. The indices Chlgreen and Chlred are related to the amount of chlorophyll in the leaves. Leaves infected with TBV have purple patterns, therefore, the amount of chlorophyll in these leaves is expected to be less. The color patterns in infected leaves are caused by anthocyanin (Mowat 1985), therefore the indices ARI, mARI and RGR, which were developed as anthocyanin indicators, might also discriminate between healthy and diseased leaves.

Spectral camera

The color camera has only a red, green and blue value per pixel, but the spectral camera gives the complete reflectance spectrum from 430 to 900 nm with a resolution of 4.5 nm. Compared to the color camera the spectral camera adds detailed spectral information per pixel. Compared to the spectrophotometer, the spectral camera adds spatial information to the spectra. The spectral imaging system is built around an imaging spectrograph from Spectral Imaging Ltd. (Specim). A detailed description of the system has been published by Polder et al. (2003). Figure 4 shows an example of spectral image data, with images at three different wavelengths as well as the raw reflectance spectrum of one pixel.

Fig. 4
figure 4

Part of the complete spectral image of a tulip leaf. Three wavelength-images only are shown. There are 257 two-dimensional images in total with a range in wavelengths of 430–900 nm. The spectra show the reflectance of all wavelengths at the position of the selected pixels in analog to digital units [adu]

The same shape features were calculated as for the RGB images. The spectral images were segmented by dividing the sum of the pixels in the spectral range 560–590 nm by the sum of those in the range 740–780 nm. The red or purple spots were quantified by a simple threshold of the resulting two-dimensional image.

Chlorophyll fluorescence imaging system

The chlorophyll fluorescence imaging system developed by Jalink (Jalink et al. 2004) provides a non-invasive technique to monitor the photosynthetic reaction to stress factors. The output from the imaging system is an estimate of the maximum quantum yield of photo system II (PS II) photochemistry

$$ \Upphi_{\text{p}} = {\frac{{F_{\text{v}} }}{{F_{\text{m}} }}} = {\frac{{(F_{\text{m}} - F_{0} )}}{{F_{\text{m}} }}}\,, $$

where \( \Upphi_{\text{p}} \) are the pixel values between 0 and 1, often denoted as photosynthetic activity (PA), F m is the maximum fluorescence at saturated light, F v is the variable fluorescence (Kooten and Snel 1990) and F 0 is the minimal fluorescence for plants with photosynthetic activity which is adapted to dark conditions. For these plants all reaction centres of PS II are open. The images were analyzed by calculating the mean and standard deviation of \( \Upphi_{\text{p}} \) for each leaf. Furthermore, thresholds were applied with small differences in \( \Upphi_{\text{p}} \), e.g. 0.4–0.5, 0.5–0.6, etc. The size of the objects after these thresholds was used in the classification. Figure 5 gives an example of such an image and segmentation results using different thresholds.

Fig. 5
figure 5

a Photosynthetic activity image, and segmented images at threshold values of: b 0.5–1, c 0.4–0.6, d 0.3–0.7, e 0.45–0.55 and f 0.4–0.5

Combination of techniques

By combining the results of the four optical methods, the performance might be improved further.

For each of the four optical methods, the classification was done for individual leaves. For each leaf and method combination, the selected class (healthy or infected) is denoted. Combined results are calculated by counting the number of healthy and infected classified leaves.

Data analysis

Each optical technique described above results in 10–40 features. These features were used in Fisher’s linear discriminant analysis (LDA), a supervised classification technique. It is based on a model that is trained using the features of known healthy or diseased leaves. The ELISA measurements were used as ground truth. The training algorithm calculates the optimal linear separation function, assuming equal (co)variance over the two groups (healthy and infected). Leave one out cross-validation was used to predict whether a plant is healthy or diseased. In leave one out cross-validation all the samples except for one are used for training and the remaining sample is used for validation. This is repeated so that each observation in the sample is used once as the validation datum. This cross-validation gives an unbiased estimate of the prediction error within the data. All analyses were done using Matlab (The Mathworks Inc. Natick, Mass, USA) and the Matlab PRTools toolbox (Heijden et al. 2004).

Results and discussion

Visual assessment

The errors of the visual assessment by the crop expert compared to the ELISA scores are 13% for the tulip variety Barcelona, 28% for Monte Carlo and 9% for Yokohama. The results for the individual plants analyzed by ELISA are given in Table 2. Columns two and three give the number of plants that were scored as healthy and diseased by both the ELISA test and the crop expert. Columns four and five give the number of plants that were diseased and healthy according to the ELISA test, but were scored as healthy and diseased, respectively, by the expert. The last column gives the total percentage error, defined as 100% times the sum of columns four and five divided by the total number of leaves analyzed. The results clearly indicate that there is a discrepancy between the ELISA technique and visual assessment.

Table 2 Visual assessment score compared to ELISA analysis (H healthy, I infected)

Color camera

Analysis of the RGB values of the Macbeth color standard in the camera images showed minimal differences between images. The illumination was consistent and no correction between images was needed. Classification was done on the shape features, on the red or purple spot features and on a combination of these. The combination of features provides the best classification of the three resulting in errors of 23% for Barcelona, 27% for Monte Carlo and 11% for Yokohama (Table 3).

Table 3 Color camera score compared to ELISA analysis (H healthy, I infected)

Spectrophotometer

The difference between the mean reflectance spectrum of the healthy and infected plants is small (Fig. 6). The difference is largest in the visible region between 500 and 700 nm (Fig. 7). The near-infrared region (above 800 nm) shows fewer differences.

Fig. 6
figure 6

The mean reflectance spectra of all healthy and TBV diseased leaves

Fig. 7
figure 7

Difference between the mean reflectance spectra of all healthy and TBV diseased leaves with the 95% confidence interval of this difference and the standard deviation of a single reflectance spectrum

The error of the classification of the individual 100 nm subspectra is also smallest in the region between 500 and 700 nm, as illustrated for variety Yokohama (Fig. 8), showing that the most important wavelengths are in the visible range (below 900 nm). This implies that for practical implementation, expensive infrared sensors are not needed which improves the economic feasibility of the system.

Fig. 8
figure 8

Classification error for subsets of 100 nm (variety Yokohama)

Two spectra were acquired for each leaf. The approximate positions of the measurement spots 1 and 2 are shown in Fig. 3. To classify the spectra independently using cross-validation, the spots 1 and 2 were treated separately and compared with the ELISA score for the whole leaf. In this way, classification results are obtained for each spot independently of the other spot. The classification error using LDA for 70 features covering the full spectral range at 30 nm interval is calculated for each spot. For the spectra of spot 1, the error is 31, 27 and 14% for Barcelona, Monte Carlo and Yokohama, respectively. For spot 2, the error is 26, 28 and 23%, respectively, for the three varieties. The individual numbers are given in Table 4. Classification of the mean reflectance spectra for spots 1 and 2 results in errors of 30, 22 and 13%, respectively, for the three varieties. Combining the two spectra in one feature vector results in classification errors of 27, 23 and 13%, respectively, for the three varieties.

Table 4 Spectrophotometer score compared to ELISA analysis (H healthy, I infected)

The spectrophotometer clearly gives the worst results for Barcelona and Yokohama (14 and 27% errors, respectively, for spot 1) indicating the need to include spatial (image) information. When the spectra of the two spots are combined by taking the mean, or by taking all the data points of the two spectra together as one feature-vector, the error of classification does not improve.

The classification errors for the six vegetation indices given above range from 16 to 45% for individual indices and 13% for the combination of all indices for the variety Yokohama (Table 5).

Table 5 Classification error for several vegetation indices for the tulip variety Yokohama

These errors are generally larger than those of the whole reflectance spectrum. Only the mARI index performs better for spot 2 (19% error in stead of 23%). When several indices are combined the errors are 13 and 20% for Yokohama spot 1 and 2, respectively (Table 5). This is slightly smaller than for the whole spectrum (Yokohama—14 and 23% errors for spot 1 and 2, respectively, Table 4), indicating that non-linear combinations of a number of specific wavelengths, as used in several indices, can improve the performance of the classification.

Spectral camera

The same procedure was applied to the spectral images as for the color images, using only shape features, spot features and a combination of these. As for the color images, the combination gives the best result for the data from the spectral camera. The errors are slightly smaller than those for the color images. For Barcelona, Monte Carlo and Yokohama the errors are 18, 29 and 9%, respectively, as given in Table 6.

Table 6 Spectral camera score compared to ELISA analysis (H healthy, I infected)

Chlorophyll fluorescence imaging system

The chlorophyll fluorescence imaging system produces images with PA (\( \Upphi_{\text{p}} \)) values between 0 and 1. A threshold was applied to segment the leaf from the background. For each leaf the standard deviation and mean PA were calculated. Furthermore, thresholds within the leaf were applied using small differences in PA in 10 steps. The size ratio of the segmented area to the total leaf area was used as a measure in the LDA classification. The total number of features used in this way was 12. These features were put into a classifier and cross-validated, as described above. The results for the chlorophyll fluorescence imaging system are errors of 37, 46 and 31% for Barcelona, Monte Carlo and Yokohama, respectively. The individual results are given in Table 7. The results indicate that this method is not suitable for the specific detection of TBV. A possible explanation for this is that this system measures overall plant stress and cannot distinguish between different sources of stress such as that caused by a virus. Another explanation is that the plant has adapted itself to the disease and hence the stress due to the virus is no longer visible.

Table 7 Chlorophyll fluorescence imaging system (CF) score compared to ELISA analysis (H healthy, I infected)

Combination of techniques

Several combinations of optical methods were investigated. The combination that involved the fluorescence camera did not improve the individual results of the other methods. The best result was obtained by taking the score that occurred most (majority vote) in the results of the spectrophotometer, RGB color camera and spectral camera. In this case the resulting errors are 19, 22 and 10% for Barcelona, Monte Carlo and Yokohama, respectively (Table 8), which is slightly worse than the spectral camera errors, except for Monte Carlo (cf. Table 6). This is because spectral imaging is already a combination of imaging data (as in the color images) and reflectance spectra. The reflectance in the NIR region, which was not available in the spectral imaging data, did not improve the classification results.

Table 8 Score using majority voting for the spectral, RGB and spectrophotometer method (H healthy, I infected, OT optical techniques)

Finally, the optical techniques were compared to the human expert assessment. For each leaf the score of the expert was compared to the classification of the optical technique. When the expert score of a leaf is equal to the classification result of the optical technique the scores are regarded as corresponding. Table 9 gives the percentages of corresponding scores for the spectrophotometer, color camera and spectral camera, showing that there is generally less than 50% correspondence. The classification error (Table 6), however, is almost equal to that of the crop expert (Table 2). This indicates that the optical system uses features other than those used by the crop expert. By incorporating expert knowledge that is currently not yet used in the analysis, it is anticipated that the optical system can be improved and will eventually outperform the crop expert.

Table 9 Comparison between manual assessment and optical techniques

Overall results

The best result for the optical methods is for the spectral camera (Table 6) with 9, 18 and 29% error for the respective varieties. This is only slightly higher than the score of the visual assessment by the expert.

The overall results for Monte Carlo were poor compared with the other two varieties, which was largely due to hail that caused severe damage to the leaves during the growth of these plants. Also the number of infected plants for this variety was too small for valid statistical analysis. The overall results for Yokohama gave the best results of all the tested optical techniques. This is somewhat surprising, as the yellow colored cultivars such as Yokohama are more difficult to assess optically for viral symptoms compared to the red colored cultivars such as Barcelona. There were no severe symptoms, such as leaf distortion or stunted growth, in the infected plants. Compared to the spectral camera, the color camera performed slightly worse for Barcelona and Yokohama (11 and 23% errors, respectively), showing the potential benefit of the spectral camera which provides more spectral information.

In this study the results of the ELISA measurements were taken as the reference, as this serological method has proved to be reliable, sensitive and reproducible (Derks et al. 1982). Detection of TBV using the reversed transcriptase polymerase chain reaction (RT-PCR) might provide an additional confirmation of the presence of viruses, and it can also be used to detect viruses other than TBV (Dekker et al. 1993). This will be needed in future as other viruses might be present in tulips that could be identified by the optical techniques. This would give rise to false-negative results because they are not detected by ELISA. Specific antisera are also available in ELISA to detect tulip virus X (TVX) and cucumber mosaic virus (CMV) in tulips (Beekwilder et al. 2008; Niimi et al. 2003).

Conclusion

The spectral system performed the best of the four optical systems tested in this research and gave results similar to that of visual assessment by experts. The NIR region of 900–2500 nm did not show a clear difference in the spectra between healthy and infected plant material. The use of spatial (image) information is clearly helpful as the color camera and the spectral camera gave better results than the spectrophotometer.

The fact that the spectral camera performs similarly to the crop expert is promising and justifies further research with an autonomous robot for the detection and removal of diseased tulip plants as a final goal.

Although an RGB color camera is easier to use and it performed reasonably well in the laboratory test, we have opted for the spectral camera for a field test in 2009. The reason for this is that field conditions are more difficult to control and therefore we decided to use the best performing camera system.

The application of this autonomous robot system will reduce the amount of insecticides and the considerable pressure on labor for selecting diseased plants by the crop expert.