4.1 Wind tunnel testing

In order to serve as a reference for the scatter of the data for the manoeuvres investigated, Fig. 13 shows the data collected on FliTePlat through a test campaign at the R.J. Mitchell wind tunnel, investigating the drag of the airframe in different configurations. Each point represents data taken at a sampling rate of 1kHz, averaged over 20s. This data also highlights the effect of stationary propeller position on the drag coefficients, with the vertical, horizontal and diagonal propeller position data spanning around 20% of the average ${C_{\rm{D}}}$ (we have also included the ‘no propellers’ and ‘windmilling propellers’ cases to serve as extreme reference values). The propeller position is typically not controllable for unpowered methods (such as the glide slope and slow down methods) and results in potentially significantly different drag coefficients.

Table 2. Comparison of coefficients generated through different methods with reference to the wind tunnel data.

Figure 13. FliTePlat lift and drag obtained in the wind tunnel.

The coefficients are extracted through linear regression through both the lift and drag data, with the intercept being the zero angle-of-attack lift coefficient and zero lift drag coefficient, respectively, and the resulting gradients being the lift curve slope and the lift induced drag constant. As the wind tunnel data is collected in ideal test conditions, there is little noise present in the data. The coefficients are presented in Table 2.

The data presented here serves as a ‘clean’ baseline (rarely achievable through flight testing). The following section compares this to the equivalent data collected through powered and unpowered flight testing. This also serves to compare the flight test methods with one another in order to assess not only the spread of the data but also the angle-of-attack range covered through each manoeuvre.

4.2 Battery depletion methods

The experimental process for calculating lift and drag via the basic battery method consists of flying a fixed altitude mission at a fixed airspeed, while monitoring the battery voltage (this continuous voltage monitoring and thus performance assessment increases the efficiency of data acquisition compared to unpowered manoeuvres, though generally at the expense of the breadth of the angle-of-attack coverage). For greater accuracy and repeatability we flew the automated version of the manoeuvre to obtain the results shown here; nevertheless, the resulting zero lift drag coefficient was still a substantial underestimate of that obtained from the wind tunnel data. Part of the suspected reason for this (and for the inaccuracies of the method in general) is that visual line of sight rules mean that the aircraft spends much of the flight time manoeuvring, whereas the calculations assume level flight. Figure 14 shows two typical flight test paths, with (a) maximising the turn radius subject to environmental constraints, and (b) maximising the linear segment length at the cost of a tighter turn radius.

Figure 14. Different flight paths for endurance testing.

Figure 15 presents the lift and drag curves calculated through four battery depletion flights flown with cruise speeds of 13, 15, 20 and 25m/s, respectively. These flights lasted between 42 and 116min (these duration being an additional reason why manual depletion tests are generally not practical). The angle-of-attack range that can be covered through this method is limited to angles of attack typical of the practical cruising speed range of the aircraft.

Figure 15. Lift and drag curves estimated via the basic battery depletion method.

Another drawback is the reliance on generic state of charge models. Moreover, the powertrain efficiency is estimated based on legacy data, which generally does not account for ambient temperature and battery age effects.

Similarly Fig. 16 shows the lift and drag curves calculated through two separate flights, both with indicated airspeeds ranging from 13 to 30m/s. The lift curve slope is lower than for the wind tunnel regression which increases the gradient of the drag curve. Both the basic battery depletion method and the improved battery depletion method shows the same trends when compared to the wind tunnel regression data. Due to the shorter flight times involved in the data gathering for the improved battery depletion method when compared to the basic battery depletion method, a larger angle-of-attack range can be covered. However, as the drag curve shows, the method is susceptible to noise in the data, and systemic errors such as voltage recovery in the battery when transitioning from high airspeed flight to lower airspeed flight.

Figure 16. Lift and drag curves estimated via the improved battery depletion method.

Flying multiple flight segments with varying airspeeds assumes that each segment is flown using a battery with the capacity equal to the energy expended during that segment. The data was analysed first by estimating the SOC based on Zhang et al. [Reference Zhang, Xia, Li, Cao, Lai, Zheng, Wang, Wang and Wang19], then the charge using the Equation (5). The true airspeed dependency of the powertrain efficiency must also be taken into account. Another challenge in implementing this method is the voltage recovery during the flight, when moving from high current draw at high airspeed to low current draw at low airspeed [Reference Boker, Henzinger and Radhakrishna28]. Finally, the precision of commercial-off-the-shelf sensors is also an important factor as the results are based on the recorded voltages at each point.

The mission used to collect data using thrust estimation based techniques (via voltage and current sensing and via RPM) included flying the test segment at different indicated airspeeds. The flight pattern can be seen in Fig. 17. These flights were automated and lasted between 19 and 24min. Unlike in the wind tunnel, this set of experiments requires power-on flight; therefore, the propeller slipstream increases the airflow speed over the wings, affecting the lift. Moreover, wind tunnel experiments are conducted in ideal condition, whereas flight testing is susceptible to environmental conditions such as gusts and errors in flight control inputs.

Figure 17. Flight path for thrust estimation methods.

Figure 18 presents the lift and drag curves calculated through two flights with test segments at a range of different indicated airspeeds. The data presented in Fig. 18 covers the indicated airspeed range between 13 and 30m/s. The lift curve shows a linear trend, with ${C_{{\rm{L}}0}}$ overestimated and ${C_{{\rm{L}}\alpha }}$ underestimated with respect to the wind tunnel lift curve. This effect is carried on to the drag curve with ${C_{{\rm{D}}0}}$ vastly underestimated compared to the wind tunnel data.

Figure 18. Lift and drag curves calculated from voltage and current sensors onboard.

A main source of error for this process comes from estimating the propeller’s performance. When propeller data are not available, blade element momentum theory [Reference Gudmundsson29] (through software such as pyBEMT by Giljarhus [Reference Giljarhus30]) can be used for estimating the performance curves (thrust, torque, power and efficiency) of a known propeller with known geometry such as the radius, number of blades, blade chord, aerofoil section shape and twist distribution. In this work we employed the performance data provided by propellers [31]. Since the propeller data here is obtained from the manufacturer [31] as a table of discrete steps of propeller speed, it is necessary to interpolate the data into a propeller map. Throughout the same flight presented in Fig. 17, the motor RPM and the audio data were recorded. As both of these methods rely on broadly the same analysis process, once the motor RPM were determined, a set of lift and drag curves were derived. The results of the thrust estimation from propeller maps, motor RPM and the airspeed are presented in Fig. 19. These curves exhibit largely the same discrepancies with respect to the wind tunnel lift and drag curves as seen in Fig. 18.

Figure 19. Lift and drag curves for thrust estimation using IR sensor.

4.3 Averaged glide slope curves – manual and automated

The data presented here was generated with FliTePlat’s propellers electrically locked, on two different days. The manual glide slopes were preformed by an experienced UAV test pilot following the process described in Section 3.3.1 with the automated glide slopes following the method described in Section 3.3.2.

Figure 20 represents the lift curves generated by the manual and automated flights, both analysed by averaging the data over each glide slope. As can be seen, the automated manoeuvre’s error bars are smaller than for the manual manoeuvres, thus demonstrating the piloting improvement gained through automating the manoeuvre, especially at high indicated airspeed glides (low angles of attack). The two resulting curves are significantly different, with the least squares fit resulting in a ${C_{{\rm{L}}0}}$ of 0.2195 and 0.3242 for the manual and automated manoeuvres, respectively. Figure 20 represents the drag curves generated from the data. This varies substantially more than the lift curve, however, the ${C_{{\rm{D}}0}}$ generated through automated manoeuvres is 9.69% away from the wind tunnel data, whereas the ${C_{{\rm{D}}0}}$ generated through manual manoeuvres is 70.37% out, demonstrating the increased accuracy of the automated manoeuvres over the manual manoeuvres (re-iterating the caveat that the wind tunnel derived reference data itself may be subject to its own offsets from ‘reality’, but it is much more repeatable than individual flight tests). For comparisons of the remaining coefficients generated through these manoeuvres with the wind tunnel coefficients, see Table 2.

Figure 20. Glide slope manoeuvre derived lift and drag, with $ \pm $ one standard deviation error bars.

Limitations of this method include the loss in altitude throughout and, as for all unpowered methods, the need for a rapid restart capability at the end of the glide. In the UK, visual line of sight regulations limit UAV flight to a maximum altitude of approximately 120m. Factoring in the altitude required for recovering from the glide in a safe manner, this limits the typical manoeuvre altitude change available to around 70m. At an indicated airspeed of 27m/s, typical for the low end of the angle-of-attack spectrum in the case of FliTePlat, this limits the data collection time to 10s or so, with corresponding limitations in terms of accuracy. Due to the high descent rates, this method also carries considerable risk and potential loss of the airframe in case the recovery is not initiated in time. For combustion engine powered UAVs the method either needs to take into account the additional drag from an idling engine, or each glide has to end in a dead-stick landing which limits the practicality of the method.

4.4 Slow down manoeuvre

The slow down manoeuvre described in Section 3.3.3 was repeated ten times in order to reduce the errors caused by gusts. The lift and drag curves are presented in Fig. 21. Note the high angle-of-attack range achieved with this method – this is enabled through the high indicated airspeed range between the maximum level flight airspeed and stall. The data follows similar trends to the automated glide slope method.

Figure 21. Lift and drag curves for ten slow down manoeuvres.

Figure 22. Lift curve slope comparison for different methods.

This manoeuvre is very constrained by the 500m radius of visual line of sight regulations as both the acceleration to entry velocity and slow down to stall need to be performed on the same straight segment. Another issue here (shared with the descent-based methods) is that of the propellers stopped in an unknown position, adding an unknown drag increment. This method also relies on a correctly tuned autopilot, which needs to perform the manoeuvre smoothly, while maintaining altitude. The manoeuvre also carries risks in case the autopilot fails to recover from the stall or recovers in a manner that exceeds the airframe’s maximum load factor, leading to potential damage to or loss of the airframe.

4.5 Method comparison summary

Table 2 summarised the data collected through the methods above, where applicable. An interesting observation to make here is that the lift curve slopes for all flight test methods presented are consistently lower than the wind tunnel data, suggesting that the aerodynamically clean conditions in the wind tunnel do not accurately represent the effect of atmospheric conditions at this scale. The increased lift curve slope from the wind tunnel data may also be due to the effect of the wind tunnel mount used, which consisted of sheet steel aligned with the airflow. This could have acted as a wing fence, segregating the low pressure region above the wings from the fuselage. Nevertheless, the wind tunnel coefficients are still a good comparative reference to the flight test methods.

Figure 22 represents the lift curves obtained through the different methods. We plotted the results of all the methods we have tested, though some may be disregarded for practical reasons. These are the manual glide method, which is an inferior version of the automated glide slope method, and the two battery depletion methods which rely on over-simplified models and assumptions and are not as accurate as the thrust estimation methods.

There are clear distinctions between the power-on methods (blue) and the unpowered methods (red) which become apparent here. The power-on methods have a higher ${C_{{\rm{L}}0}}$ than the unpowered methods due to the additional airflow over the wings generated by the propellers and the potential flow disruption present in the unpowered methods as a result of a stationary propeller. Figure 23 represents the equivalent drag curves. Here it is clear that the unpowered methods yield a higher ${C_{{\rm{D}}0}}$ than the power-on methods due to the additional drag of stationary propellers of various orientations. During prior wind tunnel testing we found that this difference can range between 0.01 and 0.03 ${C_{\rm{D}}}$ for FliTePlat for horizontal and vertical propellers, respectively, which is consistent with the difference in ${C_{{\rm{D}}0}}$ seen here.

Table 3. Comparison of the predicted performance of aircraft with the coefficients from Table 2 with reference to the wind tunnel data.

Figure 23. Drag curve slope comparison for different methods.

So what is the practical impact of the deviations observed here, in terms of engineering and operations? Using the coefficients from Table 2 where applicable, the ideal range and endurance for FliTePlat can be calculated using the electric Brequet range and endurance equations [Reference Voskuijl, van Bogaert and Rao32]. Table 3 summarises the predicted performance calculated from the coefficients in Table 2.

Figure 24. Maximum gust speeds experienced during automated glide slope and slow down manoeuvres.

The recommended power-on methods are the voltage and current method and the thrust estimation from RPM sensors. These methods do, however, rely on an estimated powertrain efficiency for the former and a reliable propeller map for latter. For unpowered methods, both the automated glide slope method and the automated slow down method provide promising coefficients with the additional drag and airflow disturbance of the stationary propellers.

At this point the reader may be wondering why the battery depletion method is not seen as the best way of estimating endurance/range (after all, both figures imply flying the aircraft to battery depletion). There are two practical reasons for this. On the one hand, flying to 0% state of charge is likely to cause damage to a potentially expensive battery or, at the very least, to affect its capacity from one test flight to the next (leading to a decline in performance through the test). The other disadvantage of simply flying to depletion (when compared to those methods that yield a more accurate estimate of the drag coefficients) is that the results thus obtained cannot be extrapolated to different mission profiles (for example ones requiring multiple climb and descent segments). Thus, the range and endurance figures shown in Table 3 should simply be read as more convenient, intuitive numbers to compare than the aerodynamic coefficients they were derived from, rather than specific estimates of range and endurance.

4.6 Manoeuvre strategy optimisation

The automated, unpowered glide and slow down methods emerge from the foregoing as having a number of practical advantages, as well as reliable accuracy. We have also found that their practical usefulness may depend on the specific environmental conditions on the day and that each has its drawbacks and pitfalls. All this raises the question of how one might construct a robustly accurate test campaign that mixes these two methods in an optimal way, for example to maximise the accuracy of the estimates of aerodynamic coefficients within a given time budget.

In this section we will answer this question using data obtained through 45 glide slope and 55 slow down manoeuvres, flown over a single multi-day flight test campaign using FliTePlat.

The flights were performed in two distinct sets of weather conditions (Fig. 24), characterised by wind speeds below 2m/s and above 4m/s, respectively. From Fig. 24 it is clear that the automated glide slope and slow down manoeuvres experience different gust magnitudes associated with various mean windspeeds. This is due to the fundamental differences between the two manoeuvres – the automated glide slope method involves descents through the atmospheric boundary layer, whereas the slow down method is performed at a constant altitude. Further, the glide slope manoeuvres may take longer to complete, which likely explains the greater scatter of the ‘+’ points on the figure.

In what follows, we shall examine a range of strategies, that is, combinations of glide descents and slow down segments. Each individual strategy (combination of descents and slow downs) is evaluated to generate a set of lift and drag coefficients ( ${C_{{\rm{L}}0}}$ , ${C_{{\rm{L}}\alpha }}$ , ${C_{{\rm{D}}0}}$ and k) from the data observed during these manoeuvres. We employ a Monte Carlo approach here, which, at each iteration, randomly samples (with replacement) half of the data points, and calculates the coefficients corresponding to this subset (bootstrapping [Reference Hastie, Tibshirani and Friedman33]). We perform 100 such iterations to generate a set of sets of coefficients (100 sets of ${C_{{\rm{L}}0}}$ , ${C_{{\rm{L}}\alpha }}$ , ${C_{{\rm{D}}0}}$ and k), which enables calculating a mean and a standard deviation for each coefficient. Taking three manoeuvres as an example (Fig. 25), these can either consist of three glide slope manoeuvres ((a) and (b)), three slow down manoeuvres or a combination of manoeuvres (two slow down and one glide slope or one slow down and two glide slope (c) manoeuvres). Additionally to selecting the combination of glide slope and slow down manoeuvres, the glide slope manoeuvres require a good range of airspeeds to increase their coefficients’ accuracy as is evident through comparing Fig. 25 ((a) and (b)). Hence, in choosing an optimal strategy for each number of manoeuvres available, it is important to determine both the combination of glide slope and slow down manoeuvres as well as the indicated airspeeds for the glide slope manoeuvres.

Figure 25. Lift curves demonstrating the importance of having a good spread of glide airspeeds and the benefits of combining manoeuvres.

As a ‘ground truth’ for comparing various subsets, we used the data collected from all 100 automated glide slope and slow down manoeuvres. This is used to work out a normalised mean error between the mean coefficients generated from the Monte Carlo simulation and the ‘true’ coefficients.

The objective function chosen for this task is a combination of the weighted sum of the error in the mean and combined standard deviation. The error in the mean requires a ground truth coefficient set to be known, used to calculate the difference of each coefficient in the subset to the accepted mean. The errors are then normalised by the accepted mean value for each coefficient and summed to provide an indication of the accuracy of the lift and drag curves:

\begin{align*}{\hat \mu _{e,i}} = \frac{{{C_{{\rm{D}}0,i}} - {C_{{\rm{D}}0,{\rm{true}}}}}}{{{C_{{\rm{D}}0,{\rm{true}}}}}} + \frac{{{k_i} - {k_{{\rm{true}}}}}}{{{k_{{\rm{true}}}}}} + \frac{{{C_{{\rm{L}}0,i}} - {C_{{\rm{L}}0,{\rm{true}}}}}}{{{C_{{\rm{L}}0,{\rm{true}}}}}} + \frac{{{C_{{\rm{L}}\alpha ,i}} - {C_{{\rm{L}}\alpha ,{\rm{true}}}}}}{{{C_{{\rm{L}}\alpha ,{\rm{true}}}}}},\end{align*}

where ${\hat \mu _{e,i}}$ is the mean error for coefficient set i.

The standard deviation is, similarly, normalised by the accepted coefficients and is then summed,

\begin{align*}{\hat \sigma _i} = \frac{{{\sigma _{{C_{{\rm{D}}0}},i}}}}{{{C_{{\rm{D}}0,{\rm{true}}}}}} + \frac{{{\sigma _{k,i}}}}{{{k_{{\rm{true}}}}}} + \frac{{{\sigma _{{C_{{\rm{L}}0}},i}}}}{{{C_{{\rm{L}}0,{\rm{true}}}}}} + \frac{{{\sigma _{{C_{{\rm{L}}\alpha }},i}}}}{{{C_{{\rm{L}}\alpha ,{\rm{true}}}}}},\end{align*}

where $\sigma $ is the standard deviation of the respective coefficient and ${\hat \sigma _i}$ is the summed normalised standard deviation of the strategy.

The weighted objective function used to compare different strategies equally considers the sum of the normalised mean error and sum of the normalised standard deviation. The sum of the normalised mean error represents a measure of the accuracy achievable through following a strategy and the sum of the normalised stand deviations represents a measure of the precision. The objective function is therefore defined as

\begin{align*}{\rm{Objective\;function}} = {\rm{\;}}{w_1} \times \left( {\sum {{\hat \mu }_{\rm{e}}}} \right){\rm{\;}} + {\rm{\;}}{w_2} \times \left( {\sum \hat \sigma } \right),\end{align*}

where ${\hat \mu _{\rm{e}}}$ is the normalised mean error, $\hat \sigma $ is the normalised standard deviation, ${w_1}$ and ${w_2}$ are weights and ${w_1} + {w_2} = 1$ .

The search space is explored by creating a grid of template strategies for each number of manoeuvres (e.g. two glide slope and one slow down manoeuvre). These templates are then used to create n strategies by randomly populating the template with non-repeating manoeuvres from the manoeuvre pool. Each strategy is evaluated using the Monte Carlo process and the objective function.

This process relies on real flight data, which is subject to random (as well as systematic) errors. As a result, each individual manoeuvre may perform slightly better or worse than anticipated. A gust may cause a glide to sweep through a larger range of angles of attack than expected, or may upset a slow down through rapid changes in airspeed. Hence, the optima found through the heuristic described above may not be robust, that is, they may perform very well in a specific set of conditions, but they may not generalise well across a range of conditions. Thus we need to find robust strategies that maximise the accuracy of the estimates of aerodynamic coefficients within a given time budget.

We achieve this by taking the previously calculated strategies for each number of manoeuvres and evaluating the strategies for their dependence on individual manoeuvres (robustness). We created new sub-strategies by first replacing each glide slope manoeuvre with a new glide slope manoeuvre in the same airspeed range. This is done twice to double the number of sub-strategies – for a strategy with two glides, this would result in four sub-strategies. Each sub-strategy is then evaluated three further times with the slow down manoeuvres replaced with random slow down manoeuvres. An example of this process using an arbitrary four manoeuvre strategy is shown in Fig. 26 where first the glide slope manoeuvres are replaced to generate the initial sub-strategies and then the slow down manoeuvres are replaced for each glide slope configuration. This example would result in 12 sub-strategies. The resulting sub-strategies’ objective functions (as determined through the Monte Carlo process) can then be averaged to get a robust estimate of the objective function of the strategy.

Figure 26. Robust optimisation process visualisation.

The resulting optimal strategies are presented in Figs 27 and 28 for the low windspeed and high windspeed sets respectively. The glide slope manoeuvres are represented by the appropriate gliding airspeed and the slow down manoeuvres are represented by SDx where x is an identification number. Pareto fronts are a useful tool for displaying the results of multi-objective optimisation, in this case, minimising the number of manoeuvres and minimising the robust objective function. Going from left to right, the first point is the absolute minimum for the number of manoeuvres and the optimum robust objective function value for this. Each subsequent point increases the number of manoeuvres but decreases the robust objective function value. Hence, the Pareto front presents a series of optimal objective function combinations.

Figure 27. Pareto front for optimal strategy determination – low windspeed data.

Figure 28. Pareto front for optimal strategy determination – high windspeed data.

The figures demonstrate that each windspeed condition has a different optimal strategy set associated with it. The low windspeed condition (Fig. 27) yields slow down only strategies. The high windspeed condition (Fig. 28), in contrast, starts with a slow down (by necessity, as a single manoeuvre strategy can only be one that sweeps the entire alpha range), followed by a mixed two-manoeuvre strategy, and then switches to glide slope only strategies. Note also how the robust optimisation yields strategies that vary the gliding airspeed; this is expected, as it then facilitates the uniform coverage of the indicated airspeed range (recall that FliTePlat’s minimum and maximum safe gliding airspeeds are 12 and 27m/s, respectively).

From this, it is clear that in windy conditions the accuracy of the glide slope manoeuvre is more robust, but in calm conditions superior accuracy is observed when flying slow down manoeuvres. This is because, while it is more efficient at covering the required range of angles of attack, the slow down manoeuvre is more susceptible to disturbances.

So how well are these results likely to generalise across the aircraft scale spectrum? While the manoeuvres themselves are scale-agnostic, the experiments we conducted here focused on the case of small aircraft limited to a 500m operating radius (within visual line of sight) and planetary boundary layer operations (below 400ft). This means that the results are only applicable to low altitude and limited area operations, which are characterised by ground-atmosphere interactions (turbulence) and by limited availability to straight ground track segments and glide descent lengths.