3.1 General presentation of the software and its workflow

ACORBA is a minimalist and user-friendly program written in the Python programming language. The software implements a fully automated workflow allowing the user to measure primary root bending angle from images obtained on a microscope or flatbed scanner. ACORBA can also be used as a semi-automated approach with manual annotation of roots from various inputs such as cameras, stereomicroscopes, and mobile phones.

The workflow is divided into two main steps. First, the root surface is identified in the image by image segmentation creating a binary mask isolating the root surface(s) (and root tip(s) for microscopy images). To improve versatility, ACORBA has three possible segmentation implementations: (a) traditional image segmentation by automatic thresholding, (b) deep machine learning prediction models and (c) the possibility to use binary masks created by the user. Roots on scanned images can be challenging to segment so we added a second traditional method and a super accuracy mode for deep machine learning. The user can decide which approach is the best for a given dataset with an implemented option called ‘test segmentation’. Second, the software automatically calculates the angle of the root tip(s). The approach used in ACORBA is the creation of a Euclidean vector that originates in the centre of the root meristem and points in the direction of the actual root tip. With this method, the root tip angle can be determined in a 360° space. To facilitate interpretation, the angles are exported in a +180/−180° space corresponding to downward/upward gravitropic bending or left/right root waving, respectively.

ACORBA includes various options including the possibility for the user to export the segmentation without angle analysis and use his own deep machine learning models. These models have to follow rules described in the user manual (Supplementary User Manual). Uploading custom prediction models to ACORBA can be used for researchers who want to analyse another species or have a large number of very specific images.

Data output-wise, the user has the option to normalize the bending angle data to the initial angle of each root. In other words, the first angle of one set of bending angles (for one root) can be automatically subtracted from the rest of the time frames to obtain relative bending angles. The raw angles are always exported as well. Users can also save the analysis plots which appear during the analysis and show the vectors. These options are particularly helpful for troubleshooting. Finally, to allow debugging any unusual output, the user has the possibility to save all the root predictions over time to check if the segmentation method selected is accurate for the whole stack of frames.

3.2 Establishment of image/mask libraries and neural network architectures

Deep machine learning models are trained using libraries containing original images and their corresponding ground truth images. The lasts are binary masks (black and white pixels) in which the element to be predicted was manually annotated.

Recently, we developed a method to observe the root gravitropic response on a vertical stage microscope involving limited mechanical stress compared with the classical method of sandwiching the root between a layer of agar and the coverglass (Serre et al., Reference Serre, Kralík, Yun, Shabala, Slouka and Fendrych2021). In this method, which we will refer to as the ‘Through’ method, a thin layer of ½ MS agar is casted directly on a microscopy chamber coverglass, and the seedlings are placed on the top (Figure 1a). The Through method is adapted to experiments in which the user is not interested in fluorescence signals while the standard Sandwich method allows classical imaging of root fluorescence.

Figure 1. Establishment of the deep machine learning libraries. (a) Comparison of the Through and Sandwich methods. (b) Establishment of the microscopy libraries. From original images, the roots were annotated to create binary masks. Then, the original images were augmented (cropped, scaled up, rotated) and reduced to 256 × 256 pixels images. (c) Establishment of the flatbed scanner library. From original images, the roots were annotated to create binary masks. Then, the original images and their masks were padded with a black border to a size divisible by 256. Finally, the last images were divided into 256 × 256 tiles.

We created libraries and trained root surface prediction models for both microscopy methods. In parallel, we also trained root tip area detection models. Indeed, the estimation of the middle line of the root for the angle calculation requires a skeletonization step which is more accurate if restricted to a smaller area such as the root tip. Both predictions are relevant as the root surface, as a whole, is a defined structure in a picture while the root tip area is a subjective zone. The subjective aspect of the root tip area prediction makes it untrustworthy for determining the root angle directly. The original libraries created for both microscopy methods consisted of 302 and 189 images/masks for the Through and the Sandwich methods, respectively. These numbers are in general considered small for deep machine learning training as they are unlikely to display enough diversity to obtain a prediction model with a wide range accuracy. To mitigate this problem and artificially increase the diversity and the final model accuracy, we used a method called image augmentation in which the original images and their masks are duplicated and then modified (rotation, cropping, Figure 1b). The final libraries contained 785 and 757 images/masks for the Through and Sandwich methods, respectively. For the root tip prediction models, the images were not cropped to avoid losing the overall context of the root and increase accuracy. All the models were trained with images/masks resized to 256 × 256 pixels for faster training and analysis. The images were obtained from several microscopy cameras (see Materials and Methods).

For images obtained with a flatbed scanner, the process was different. On the one hand, skeletonization of long and thin objects is comparably easy, so we only trained a root surface prediction model. On the other hand, images of scanned square plates are large with small objects to be detected (at 1200 dpi, the root thickness is approximately 10 pixels), and the images contain cotyledons and hypocotyls which are not relevant to the analysis. Therefore, cotyledons and hypocotyls were manually cropped out of the images, only leaving root parts (Supplementary Figure S1a). It was not possible to simply resize the raw images to 256 × 256 padded squares as this produced extreme inaccuracy in angle measurements. To circumvent this, the raw images of various original sizes were first padded to a rectangular image that can further be divided into 256 × 256 tiles (Figure 1c). The pre-library contained 340 original images/masks with 5–30 roots in one row obtained using commonly used Epson flatbed scanners (see Materials and Methods). The library was augmented (horizontal flip and contrast adjustments) to a final library containing 21,816 images/masks containing background or pieces of roots.

The neural networks trained in this study are modified UNET architectures (Ronneberger et al., Reference Ronneberger, Fischer and Brox2015) (Supplementary Figure S1b). This architecture is commonly used for its efficiency for semantic segmentation in biology. The network was implemented using TensorFlow implementing Keras, https://github.com/tensorflow) and the package Keras-UNET (https://github.com/karolzak/keras-unet). The super accuracy mode for scanner images is using the same prediction model. However, the prediction is averaged from mirrored and rotated pictures to increase accuracy. We used the ADAM optimizer (Kingma & Ba, Reference Kingma and Ba2017) and the Dice coefficient metric to quantify the prediction accuracy over the training periods. The accuracies of our trained models are presented in Supplementary Table I. The root surface prediction accuracies for the microscopy methods were above 93% (For comparison with manual annotations, see Supplementary Figures S1–S4). Given the size of the root surface, these scores are more than satisfying to allow accuracy in further angle measurements. The accuracies for the root tips were lower, as expected. Indeed, the root tip area is not a defined area and was subjectively annotated. Overall, the trained models managed to recognize the pixel pattern created by the root tip shape. Furthermore, the root tip area detection is only intended to grossly detect the root tip in space. The scanner root surface prediction accuracy was 85% (Supplementary Figure S5). However, consistent manual annotations of thin objects are difficult. Most of the time, the final model predictions were visually more accurate than the manual annotations (for comparison with manual annotations, see Supplementary Figures 5–10). Deep machine learning model accuracies were also assessed using a test dataset by comparing manual and prediction segmentation of images unknown to the models. The accuracies of our models on test datasets are presented in Supplementary Table I, which showed similar accuracies than on the libraries, with the exception of root tip predictions which were lower. These root tips were annotated separately and thus, the subjective annotations of the root tips might have been different. Regardless of the differences, the performances of the models were visually more than satisfactory (https://doi.org/10.5281/zenodo.6410167).

3.3 Traditional image segmentations

As deep machine learning is unlikely to be accurate in every condition, we developed an alternative automatic solution using traditional image segmentation approaches. However, for microscopy, the root tip detection method is always conducted by deep machine learning as we could not find a satisfying method to detect the root tip area by traditional methods. For the images obtained by the Through method, an automatic Otsu threshold is applied followed by a binary closing function to fill the possible holes in the binary thresholded image (Figure 2a). For the Sandwich method, the image goes through a texture detection filter based on entropy (complexity of the grey levels in a given area). Then, the image is automatically thresholded, and a binary closing function is applied to fill the possible holes (Figure 2b). It is worth noting that unlike in case of the deep machine learning prediction, detached root caps and other objects with sufficient contrast are also segmented and could create angle measurements artefacts (Figure 2b).

Figure 2. Image segmentation in ACORBA. (a) Traditional automatic segmentation of root in vertical stage microscopy using the Through method. (b) Traditional automatic segmentation of root in vertical stage microscopy using the Sandwich method. (c) Traditional automatic segmentation of root imaged with a flatbed scanner. (d) Example of the function test segmentation for vertical stage microscopy using the microscopy Through method. (e) Example of the function test segmentation for scanner images.

For scanned images, we implemented two methods similar to both methods in microscopy (Figure 2c).

Deep machine learning predictions and traditional methods can be compared using the ‘test segmentation’ option to decide which one is the most suitable for a given dataset (Figure 2d, e). In case none of the automated methods are satisfying, the software allows the users to import their own binary masks for root surface (and root tip for microscopy). Ways to create those masks are presented in the user manual. Using custom binary masks avoids any bottleneck from image segmentation.

3.4 Calculation of the angle for microscopy and flatbed scanner images

Even though the angle calculations are based on a root tip vector for both microscopy and scanner images, the size of the root surface and the number of roots in every image is different. Determining the root tip vector for microscopy images is a more complex process than for scanner images.

Once the segmentation of roots in microscopy images is carried out, both root surface and tip masks are preprocessed before the actual determination of the root tip vector. The first step is a series of binary pixel dilation and erosion to remove potential non-specific pixel detection conducted on both root tip and root surface masks. Then, the particles are analysed, and only the largest surface (root surface/tip) is kept for analysis. Finally, to ensure that the angle orientation in space is similar for every root, the root tip is automatically oriented towards the left side of the frame.

To focus the analysis on the root tip and not on the rest of the root, the root tip area is isolated. For this, a circular region of interest (40 pixels width by default, customizable) around the centroid of the root tip mask is created (Figure 3a). This circular selection is used to crop and isolate the root tip area on the root surface mask. Finally, this cropped area is enclosed into a circular bounding box (using the circle() function in Python OpenCV package), and the root tip area perimeter is determined (Figure 3b).

Figure 3. ACORBA measurements of root tip angles. (a)–(e) For microscopy images and (f)–(I) for flatbed scanner images. (a) Cleaning of binary masks and determination of the approximate position of the root tip on the root surface prediction. (b) Isolation of the root tip on the root surface prediction and determination of the intersecting points with an enclosing circle to estimate the root tip direction. (c) Skeletonization of the root tip surface. (d) Determination of the middle of the skeleton corresponding to the angle vector origin. (e) Determination of the direction of the angle vector by modelling a linear regression between the estimated root tip direction (see b) and the angle vector origin. The intersection between the linear regression modelling and the root perimeter gives the direction to the vector. (f) Original flatbed scanner tif stack. (g) Binary mask. (h) Individual skeletonization of every root and determination of skeleton origin and end. (i) Determination of the vector origin and direction.

The creation of the vector starts by first determining the central pixel position of the root tip area. The result of enclosing the root tip perimeter into a bounding box is the creation of two intersecting pixel clusters present both in the bounding box and the root tip area perimeter: (1) the actual root tip and (2) the root tip area shootward (Figure 3b). These two clusters are identified by K-mean clustering. The smallest cluster central coordinates are kept for further use as they correspond to an estimated root tip position (Figure 3b).

Second, the software goes back to the isolated root tip surface area for skeletonization (Figure 3c). The skeleton pixels are ordered from the closest to the previously determined root tip cluster to the shootward side by a closest-to-closest pixel approach. From this ordered single line of pixels, the middle pixel is set as the origin of the vector for angle calculation (Figure 3d).

Finally, the root tip pixel’s (vector’s direction) precise location is determined by creating a straight line from the middle of the skeleton to the image frame (Figure 3e). The intersection between this line and the root perimeter is set as the root tip pixel and vector’s direction.

From this vector, the angle of the root tip is determined with the formula to determine the angle of a vector in a 360° space. However, to facilitate interpretation, 180° is automatically subtracted from the angle resulting in arbitrarily negative values for upward bending and negative values for downward bending.

This process is carried on every time frame of a stack (Supplementary video 1), and the angles are compiled into a table for export.

Creation of root tip vectors from thin and long surfaces is simpler than from large surfaces. However, working with several objects in one image creates new challenges that we describe below.

Once the full binary mask is reconstructed from the 256 × 256 segmentation patches (Figure 3f, g), the mask size is reduced by 40% to increase analysis speed without significantly decreasing accuracy.

The first step of the analysis is to detect every root in the images and isolate them as individuals before skeletonization which represents the roots as a single line of pixels (Figure 3h). Next, both ends of the skeleton are detected (Figure 3h). The identification of the shootward end and the root tip is facilitated by the mandatory step required from the users to orient the root tips towards the right side. The shootward end is determined as the leftmost coordinate and the root tip is identified as the second skeleton end. This step is followed by the reorganization of the skeleton pixels from the shootward end to the root tip by a closest-to-closest coordinate approach.

Finally, the angle vector is determined individually for each root tip. The root tip direction is determined as the root tip end identified above. The vector’s origin is set, by default (customizable), as the 10th pixel on the skeleton starting from the end of the root (shootward) (Figure 3i).

We previously stated that the segmentation of several thin objects in a large image can come with challenges such as a not fully segmented root which appears as two roots. To mitigate this and ensure that all the roots are in one piece, a function calculates the distance between all the skeleton ends and origins. If one skeleton end is less than 50 pixels (by default, customizable) from another skeleton origin, the software assumes that these two pieces belong to one root and those two pieces are linked to recalculate the angles.

Similarly, to the microscopy images, the whole process from reconstructing the images from tiles to the calculation of the angles is carried out in every time frame (Supplementary video 2), the angles are then compiled into a table for export.

3.5 Evaluation of microscopy models and method

To assess the accuracy of the angle calculation method for microscopy, we first compared manual and automated analyses of the same roots bending over time in a vertical stage microscope. For the manual analysis, we used the manual equivalent of ACORBA’s method (Supplementary Figure S4). ACORBA and manual measurements, overall, produced comparable results (Figure 4a, Root 1 and 2). However, in some cases, manual measurements produced significant differences between repeated measurements of the same roots, in contrast to ACORBA which always produced the same result for one root (Figure 4a, Root1 and 4). This demonstrated that ACORBA is more reproducible than the human eye and hand in manual measurements. Moreover, the analysis time by ACORBA is roughly 10–20 times faster (depending on the computer hardware) than the manual analysis and does not require oversight.

Next, to test whether ACORBA can measure angles bigger than the maximum 15° measured in our first experiment (Figure 4a), we gravistimulated and measured Col0 roots bending during the course of 5 hr (Figure 4b). We recorded angles up to 68°. In theory, the method would be able to measure angles higher than 90°. However, this scenario is unlikely to happen with a gravistimulated root observed with a vertical-stage microscope.

Further, we compared our recently published gravitropic data analysed using a different method (Serre et al., Reference Serre, Kralík, Yun, Shabala, Slouka and Fendrych2021). In this experiment, we studied the afb1 mutant impaired in the rapid response to auxin and rapid gravitropic response. We showed that afb1 was slower to trigger gravitropic bending with an approximative 12 min delay compared with the wild type Col0. The image stacks from this experiment were re-analysed using ACORBA and showed similar results (Figure 4c). However, ACORBA was faster than our previously published method as it is fully automated.

Further, to demonstrate the performances of ACORBA in another biological context, we compared the gravitropic response of Col0 and the aux1 mutant impaired in the main auxin influx carrier (Bennett et al., Reference Bennett, Marchant, Green, May, Ward, Millner, Walker, Schulz and Feldmann1996) in plants. This mutant is known to be agravitropic as auxin is not correctly redistributed during the gravitropic response. As expected, Col0 initiated bending towards the gravity vector with recorded downward angles ranging from 5 to 21° (Figure 4d). In contrast, aux1 roots displayed a typical agravitropic behaviour with growth orientation independent of the gravity vector with roots bending upward, downward or not bending, again well analysed by ACORBA (Figure 4d).

We previously described two methods used in our laboratory to assess the gravitropic response using a vertical-stage microscope (Figure 1a). Here, we used ACORBA to compare the dynamics of root bending imaged with the Through and Sandwich methods. This experiment showed that the roots imaged with the Through method are, overall, bending faster than the roots imaged with the Sandwich method (Figure 4e). This confirmed that the Through method is allowing a better gravitropic response probably by providing less mechanical stress and unobstructed growth.

To quantify ACORBA’s accuracy, we measured the root tip angles of an artificial set of roots with set angles from −89 to +89° as the microscope method is not adapted to angles bigger than 90°. These measurements showed that ACORBA can measure angles in this range with a small margin of error (in average 0.74%, Figure 4f).

This set of experiments allowed us to confirm that ACORBA can be used to accurately determine the bending angles of roots imaged with a vertical-stage microscope.

Furthermore, we measured poppy roots gravitropic response using the software in semi-automated mode (Supplementary Figure S4e,f). This experiment showed that using handmade binary masks, the software can measure bending angles of other species than A. thaliana.

Figure 4. Characterization of the angle calculation method for vertical stage microscopy images. (a) Comparison of ACORBA and manual measurements of Col0 root gravitropic angles. n = 4 individual seedlings. (b) Calculation of Col0 root gravitropic angle over 302 minutes. n = 6 individual seedlings. (c) Comparison of Col0 and afb1 gravitropic bending angles. n = 9 (Col0) and 12 (afb1) individual seedlings. (d) Comparison of Col0 and aux1 gravitropic bending angles. n = 13 (Col0) and 19 (aux1) individual seedlings. Images were taken every 2 min. (e) Effect of the imaging method (Through versus Sandwich) on Col0 seedling gravitropic angles in a vertical-stage microscope. Represented data are mean ± SD (shaded area). n = 10 (Through) and six (Sandwich) individual seedlings. (f) Accuracy of ACORBA on an artificial set of bended roots from −89 to 89°.

Figure 5. Characterization of the angle calculation method for flatbed scanner images. (a) Comparison of manual and ACORBA measurements of Col0 gravitropic bending angles. n = 9 individual seedlings. (b) Comparison of Col0 and aux1 gravitropic bending angles. n = 9 (Col0) and 8 (aux1) individual seedlings. (c) Measurement of root tip angles during horizontal root curling. (d) Accuracy of ACORBA on an artificial set of bended roots from −90 to 90°. (e) Effect of no sucrose and (f) 3% sucrose on Col0 vertical root waving. n = 10 (0% sucrose) and 11 (3% sucrose) individual seedlings. (g) ACORBA semi-automated measure with manual binary masks of root tip angles of Col0 seedling growing horizontally and observed under a stereomicroscope. Images were taken every 30 min.

3.6 Evaluation of the scanner model and method

We progressed further with the evaluation of the scanner angle calculation. We first compared ACORBA and manual measurements (method described in Supplementary Figure S4a) of root bending on agar plates measured over 15 hr after gravistimulation (Figure 5a). We showed that ACORBA and manual measurements were strikingly similar. However, ACORBA produced more noisy data. Nevertheless, in most cases the bending angles of roots in plates are averaged and the overall average and error bars between methods were almost identical. Furthermore, the analysis time is considerably shorter with ACORBA (up to 10 times depending on hardware) and does not require oversight.

Next, we also compared Col0 and the agravitropic mutant aux1 responses and showed that while Col0 bent downward up to 130° over 15 hr, the mutant bent slightly downward or upward, showing a typical agravitropic behaviour (Figure 5b).

To push the limits of the software, we imaged a root curling, a phenomenon appearing when agar plates are horizontally positioned. In this scenario, the deep machine learning segmentation showed untrustworthy accuracy above 158° bending. However, the automatic traditional segmentation method and the angle calculation method showed that ACORBA can measure relative angles up to 180°. Passing the 180° threshold up/downward bending resulted in a complex output, with difficult interpretation as the root tip is changing horizontal direction (Figure 5c). The complex interpretation is a direct consequence of working in a +180/−180° space instead of a 360° one. However, this situation is not common, and the gain in interpretation facility obtained by working in a +180/−180° space for regular gravitropic experiments overcomes this specific limitation. These results demonstrate the limitations of the software for measurements of curling behaviour, and we conclude that ACORBA should not be used for measuring curling and/or touching roots.

Further, to quantify ACORBA’s precision, we quantified the root tip angles of a set of artificial root images with set angles ranging from −90° to +90° (Figure 5d). This experiment allowed us to demonstrate that the software can measure root tip angle from scanned images with an average margin of error of 1.05%.

To demonstrate the software’s ability to measure angle of waving roots, we imaged seedlings growing vertically on 1/2MS medium without or with 3% sucrose, known to induce waving (for review, see Oliva & Dunand, Reference Oliva and Dunand2007) for 20 hr. The seedlings transferred from 1% sucrose to no sucrose showed almost straight root growth after 6 hr (Figure 5e). However, seedlings transferred to 3% sucrose medium displayed heavy waving with a waving amplitude of approximately ±60° (Figure 5f).

Finally, to illustrate the semi-automated use of the software, we imaged seedlings growing in agar plates using a stereomicroscope and a mobile phone (16 megapixels resolution) (Figure 5g). After resizing the images, so that the roots were approximately 10 pixels in width (similar to 1200 dpi scanned images), we prepared the binary masks manually in ImageJ (Method in Supplementary User manual, Figure S5a,c). It is worth noting that seedlings from the mobile phone image and the stereomicroscope were also correctly segmented by the deep machine learning model Supplementary Figure S5b,d). In these conditions, ACORBA was able to measure single time frame angle from mobile phone image (Supplementary Table II) and angle dynamic of horizontally growing seedlings observed with a stereomicroscope (Figure 5g).