3.1 Functions identification and description

FRAM’s first step deconstructs the complex sociotechnical system into ‘functions’, that represent an activity that is required to produce a certain outcome. The first step identifies these functions that are needed for everyday work to succeed and characterised by six different aspects as follows. Aspects are traditionally placed at the corners of a hexagon, which represents the function itself (Fig. 7).

Figure 7. A hexagon representing a function.

The input (I) activates or starts a function and/or is used or transformed by the function to produce the output (O), which is the result of the function. The output can be either an entity or a state change and serves as input to the downstream functions. Preconditions (P) are mandatory conditions that must exist before carrying out the function. They do not necessarily imply the function execution. The function needs the resource (R) when it is carried out, or consumes it, to produce the output. Controls (C) supervise, regulate or monitor the function. They are exemplified by guidelines, regulations or even social expectations. Temporal requirements or constraints of the function, regarding both duration and starting point, are given by time (T).

3.2 Performance variability characterisation

The idea of the second step is to characterise the variability of the functions that constitute the FRAM model, assessing how performance variability will show itself – either in the sense of how it can be observed or detected – or in the sense of how it may affect downstream functions. One way to do that is to evaluate subjectively the outputs variabilities. For instance, they could vary in terms of timing or precision. An output can occur too early, on time, too late or not at all. Regarding precision, it can be precise, acceptable or imprecise.

Instead of evaluating functions variability in a subjective way, this work uses FDM techniques to estimate the variabilities. An FDM program is designed to enhance safety by identifying airlines’ operational safety risks, based on the routine analysis of flight data during revenue flights. These data are compared against pre-defined envelopes and values, to check whether the aircraft has been flown outside the scope of the standard operating procedures.

Essentially, flight data are converted into engineering units in a process called decoding, which is normally described in a data frame layout document. The decoded aircraft parameters represent the entry point and are used for measurement calculations, which may be analysed by means of distributions or directly compared with a threshold to produce events (Fig. 8) [36]. The distributions and events are investigated to highlight any trend that could show a latent or potential risk. The FDM program is applied reactively through analysis of incidents or accidents; proactively through analysis of the airline’s activities; and predictively through data gathering to identify possible adverse future outcomes [Reference Delhom37].

Figure 8. Basic entities of the FDM program.

For ROs, EASA Guidance for the Implementation of Flight Data Monitoring Precursors [36] proposes algorithms to address hazards like the identification and quantification of unstable approaches; the recognition of high energy over the runway threshold through the measure of height and speeds; and the detection of inadequate use of deceleration devices like late or insufficient activation of thrust reverser and wheel brakes during rejected take-offs and landings. These algorithms are dependent on the establishment of thresholds as well as they are independent from each other. The risk for the monitored fleet remains unknown as the interactive effect among the hazards is not explored.

The variabilities of the FRAM functions are here estimated using algorithms that are similar to the proposed by the EASA Guidance. However, instead of focusing on events (i.e. how many times the parameter crosses the stablished threshold), the purpose is to understand how the system works on a daily basis and estimate how each function varies in the ‘real world’. Note that the aircraft systems are usually working as expected. Thus, the everyday variabilities of the functions are related with the human performance (i.e. the flight crew handling during the landing execution).

Flight data is the information coming from aircraft sensors, onboard computers and other instruments that are recorded into a crash-survivable flight data recorder (FDR) and occasionally also into easily accessible quick access recorder (QAR). In this work, FDR raw data from a typical jet is processed into Tab-Separated Values (tsv) files by an internal and non-commercial tool and analysed via R functions. R is a programming language and free software environment for statistical computing and graphics [38].

For most aircraft, FDR file format follows the ARINC 717 standard, being composed by a continuous stream of bits. ARINC 717 divides data into subframes and frames. A subframe is one second long and contains anywhere from 64 12-bit words, up to 2048 12-bit words, depending on the aircraft. A frame is composed of four subframes. Every frame contains exactly the same information (i.e. parameters distribution). The four subframes within each frame are not identical. Some parameters may be found in every subframe, while other may appear only once per frame. A parameter may be discrete (e.g. air-ground transition) or continuous (e.g. airspeed, groundspeed and height). In this case, an equation is usually set to convert the recorded set of bits into the data. For example, the airspeed may be recorded into Subframes 1 to 4 at Word 37 from bits 1 to 10 in a resolution of 0.5 knots. This means that the airspeed is recorded in a 1 sample per second (sps) and multiplied by 0.5, specifying a range from 0 to 511.5 knots. Given the FDR file in an ARINC 717 format and the map with the parameters location as stated by the aircraft manufacturer, some commercial tools are able to convert the raw data into a table as shown in Fig. 9. Schwaiger and Holzapfel [Reference Schwaiger and Holzapfel39] proposes a method for the decoding of binary ARINC 717 recorder files.

Figure 9. FDR raw data converted into a tsv file.

Following an eight (8) sps basis of the current tsv file, the analysis through R functions starts by interpolating the selected parameters to fill the gaps inside the converted flight data file, caused by a smaller sample rate. Continuous parameters are usually linearly interpolated to enhance their precision (i.e. to avoid steps among the samples). Continuous parameters that represent a position (e.g. thrust lever and flap surface position) as well as discrete parameters remain constant until the next sample.

Further steps consist of splitting the data into flights, characterised by the engines’ turning on and off, and of capturing the final approach/landing phase, starting at circa of 100 feet AGL prior to the air to ground transition. The FRAM functions outputs are then captured or computed for each flight cycle.

3.3 Variability aggregation

FRAM models the potential couplings among functions, not showing the effects of a specific scenario. The variability aggregation step focuses instead on examining specific instantiations of the model to understand how the potential variability of each function can become resonant, leading to unexpected results. It is therefore necessary to identify the functional upstream-downstream couplings. The variability of a function results as a combination of the function variability itself and the variability deriving from the outputs of the upstream functions, depending on the function type and the linked aspects type.

This step may be addressed qualitatively, based on potential for dampening performance variability ranges from +1 to +3 and for increasing performance variability ranges from −1 to −3 [Reference Patriarca, Di Gravio and Costantino24]. Yet, this step is addressed quantitatively here, through a correlation check between the FRAM functions aspects to validate the model followed by a logistic regression to describe the system output in terms of its upstream functions aspects.

Correlation is a dimensionless quantity that can be used to compare the relationships between pairs of variables in different units [Reference Montgomery and Runger40]. Spearman’s correlation coefficient is a statistical measure that assesses the strength and direction of a monotonic relationship between two variables. It ranges from −1 and 1, where 0 indicates no correlation, 1 means total positive correlation, and −1 signifies total negative one. The correlation confirms or discards the influences between the FRAM functions, validating and improving the model that was initially proposed on an expert experience basis.

Logistic regression is a type of predictive model that can be used when the target variable is binary, like acceptable or unacceptable outcomes. The logistic function, commonly referred to as the sigmoid function, describes the association between the predictor variables and the likelihood of the binary output. Its coefficients estimation is carried out using maximum likelihood [Reference Battisti and Smolski41]. The logistic regression is used in the current work to classify the cycles into acceptable or unacceptable outcome of the FRAM model, and to recognise the influence of each predictor variable in the target one in terms of probabilities.

An important problem in many applications of regression analysis involves selecting the set of predictor variables to be used. This selection is performed with the use of FRAM final model and the correlation between the functions’ outputs. Only variables that are independent from each other are used as inputs in the final model. Their independence is checked using the Spearman method.

3.4 Variability management

This last step consists of monitoring and managing the performance variability, identified by the functional resonance in the previous steps. Performance variability can lead both to positive and negative outcomes. The best strategy consists of amplifying the positive effects, i.e. facilitating their happening without losing control of the activities, and damping the negative effects, eliminating and preventing their occurrence.

Using a logistic regression, the exponential function of its coefficients is the odds ratio associated with a one-unit increase in the exposure. Odds ratios (OR) are used to compare the relative likelihood of the outcome occurrence (i.e. the runway overrun), given exposure to the predictor variables (e.g. height and speed at the runway threshold). The odds ratio can also be used to determine whether a particular exposure is a risk factor for a particular outcome, and to compare the magnitude of various risk factors for that outcome.

4.1 Functions identification and description

Figure 10 shows a diagram with the functions considered relevant to explain the performance variability of the landing procedure, while Table 1 summarises its variables. The diagram was drawn using the FRAM Model Visualiser.

The landing procedure starts before take-off, during the flight planning phase. The flight crew selects a runway to land the aircraft, whose required length is based on the aircraft ULD. The ‘To Prepare the Flight’ is a function that incorporates the landing assessment, evaluating if the designated runway is long enough to accommodate the landing given the airport altitude, environmental conditions, and aircraft status/ configuration. The runway margins, given by Equation (1), influences the touchdown point and the brake application profile, which is more aggressive in shorter runways. It is also directed related to the remaining runway at the initial point of the taxiing procedure (i.e. a groundspeed of 30 knots).

(1) \begin{align} RunwayMargins = \frac{{RunwayLenght}}{{ULD}}\end{align}

Theoretically, the landing procedure starts at the runway threshold with the aircraft at 50-feet AGL in landing configuration with the reference speed (v_ref). The ‘To Cross the 50-feet Height’ is a function that initiates the analysis. Its outputs could be the Δv_ref (i.e. the difference between the airspeed and the v_ref), the thrust lever angle (TLA), the pitch attitude and the flight path angle (FPA). The value of Δv_ref and TLA at 50-feet AGL are intrinsically correlated with Δv_ref and TLA at the runway threshold, respectively. TLA and the pitch attitude are manipulated by the flight crew during the flare. The FPA, associated with the height at the runway threshold, influences the touchdown point geometrically.

‘To Cross the Runway Threshold’ usually succeed the ‘To Cross the 50-feet Height’ in the fleet under analysis. Its outputs, height, Δv_ref and TLA, affect the touchdown point.

The ‘To Perform Flare’ function consists of the descent rate reduction to accomplish a smooth landing. It is normally performed near to the ground (less than 50-feet AGL) through the increase of aircraft pitch attitude simultaneously with the reduction of engine thrust. A firm touchdown results in a smaller touchdown point. Its output is given by the descent rate at the wheel spin (i.e. the moment that the wheels start rotating). In addition, this function has some internal variables such as the TLA and pitch attitude change. The most significant internal variable for its output is the moment that the aircraft reaches the maximum pitch in relation to the wheel spin. An earlier maximum pitch results in a lower descent rate and, consequently, a longer touchdown point.

‘To Touchdown’ outputs are the air-ground transition, the touchdown point and the groundspeed at this moment. The air-ground transition starts the functions that decelerate the aircraft (i.e. ‘To Press the Brake Pedal’ and ‘To Command Pitch’). The fleet under analysis does not have ground spoiler, thrust reverser nor autobrake systems. The touchdown point is given by distance between the runway threshold and the aircraft position at the first air-ground transition, impacting the remaining distance to the end of the runway at 30 knots directly. The groundspeed at touchdown is the resource that must be consumed during the deceleration process.

Table 1. Summary of the FRAM diagram variables

¹Not applicable

Figure 10. Proposed FRAM diagram for landing procedure with focus on ROs.

The flight crew starts to press the brake pedal as soon as the aircraft touches the ground, typically prior to the first air-ground transition. Nonetheless, the brake pedal position varies during the landing run as well as does not necessarily reach the 100% in the daily operation. The brake pedal output is, thus, composed by the mean application between the aircraft wheel spin, that occurs prior to the air-ground transition, and the landing end (i.e. with a groundspeed of 30 knots). The brake pedal parameter affects the ‘To Decelerate the Aircraft’ function indirectly, via the ‘To Apply Brake Pressure’ function, as the anti-skid system modulates the brake pressure based on the pedal position and the runway surface condition.

The landing roll should be performed with pitch command in neutral position. A pitch down command moves the load distribution from the main landing gear to the nose one, diminishing braking efficiency and affecting the aircraft deceleration.

Thus, the ‘To Decelerate the Aircraft’ receives the brake pressure mean and the pitch mean during the landing run as inputs. Note that, when available for the designated runway, the information regarding runway surface conditions is checked by the flight crew during the approach, represented by ‘To Check Environmental Conditions’ function.

The ‘To Decelerate the Aircraft’ output is the longitudinal deceleration, that associated with the runway margins and the touchdown point, compiles the remaining distance to the end of the runway when the groundspeed is equal to 30 knots (i.e. the landing end).

The system output is given by the remaining distance. According to Flight Safety Foundation, the risk of an overrun is higher if the remaining distance to the end of the runway is less than 2,500 feet when the velocity is greater or equal to 80 knots [33]. This premise is achieved with a deceleration of about 0.11 g or 1 m/s². Despite being a conservative value as the deceleration is normally higher, this value is used here to classify the system outcome as expected or not. Our landing ends at 30 knots as this is the taxiing speed, and the aircraft usually does not stop prior taxiing. A remaining distance of 2,500 feet at 80 knots is equivalent to 400 feet at 30 knots, considering the deceleration rate constant during the landing run. Thus, a remaining distance over 400 feet at the current landing end is considered acceptable (i.e. equal to 0) while a distance equal or under 400 feet characterises an event (i.e. equal to 1). The remaining distance is the binary variable of the current logistic regression.

4.2 Performance variability characterisation

The analysis continues by capturing the behaviour of each FRAM function aspects. Looking forward, the correlation between these variables gives preliminary tips regarding the system performance.

For example, height is one of the ‘To Cross the Runway Threshold’ outputs and its variability is shown in Fig. 11. The threshold is usually crossed under 50-feet AGL (95% of the flights), having a median of 26 feet. This output does not have any correlation with the FPA, indicating that the approach path is being displaced to reduce the required landing distance.

Figure 11. ‘To Cross the Runway Threshold Output’ behaviour.

Some aspects like height, TLA, brake pedal position, brake pressure and pitch attitude are directly captured from the FDR. Others must be estimated using the recorded parameters as basis (Table 1). For instance, the designated runway is intercepted by the minimum distance between the aircraft position at 50-feet AGL, provided by its linearly interpolated latitude and longitude, and its threshold, given by an external database. This database contains circa of 23,000 runways, specified by their airport ICAO Code, direction, length, aiming point position, width, altitude, threshold latitude and longitude.

Speed increment (Δv_ref) is given by actual airspeed minus v_ref, which depends on ice conditions, aircraft gross weight and flap position. Their relationship is available at the AFM in a table format.

The runway surface condition is an index created for this work and exhibited in Equation (2), measuring the interaction between the aircraft and the runway through the anti-skid actuation. This interaction depends on the runway surface conditions as well as tire pressure and condition, besides other attributes. There is a curve that relates the brake pedal position with the expected brake pressure, which is dependent on the brake control system under analysis. However, the designed pressure is usually not reached due to the anti-skid protection as shown in Fig. 13. In a good interaction, the actual brake pressure is close to the projected one. In a bad one, it is much lower. The index varies from 0 to 1, being the latter an indication of an ideal interaction.

(2) \begin{align}Runway\;Surface\;Index = \frac{{Current\;Brake\;Pressure}}{{Expected\;One}}\end{align}

The ‘Brake Application Profile’ is given by the brake pedals mean application between the aircraft wheel spin and the landing end as the brake pedal position varies during the landing run (Fig. 12). The mean comprises, indirectly, the moment that the flight crew starts the brake pedal application, the moment that they reach the maximum value and the maximum pedal value by itself.

Figure 12. An example of brake application profile.

Figure 13. Some FRAM model functions outputs.

Figure 13 displays the variabilities of some aspects of FRAM functions. An adequate use of the deceleration devices refers to a brake pedal profile near 100%. The pitch command on ground is neutral at zero.

The analysis of each output improves the understanding of the landing procedure, being used for the FRAM diagram adjustment.

4.3 Variability aggregation

The system output is given by the remaining distance, which is composed by the margins available on the designated runway, the touchdown point, the aircraft energy at the touchdown moment (i.e. the groundspeed) and the deceleration rate. The latter is affected by the runway surface condition index, the brake application profile and the pitch applied on ground.

Based on this set of predictor variables, the logistic regression was built by the glm R function (Table 2) [38]. The ‘Runway Margins’, for example, has a coefficient of −7.334201. A negative coefficient means that when the runway margins increase, the chances of a critical landing decrease (i.e. remaining distance under 400 feet at 30 knots). Likewise, it is noted that there is statistical significance at p = 0 in the use of this variable in the model, showing that it is important to the proposed regression model.

Table 2. Summary of the logistic regression

A more enriched interpretation of the relationship between predictor variables and the logistic regression output is the OR, built by the logitor R function from mfx package [Reference Fernihough42]. To illustrate, for each unit increment in the ‘Groundspeed at Touchdown’, the chances of a critical landing increase circa of 13%. Or, for each unit increment in the ‘Brake Application Profile’, the chances of a critical landing are reduced by about 49%.

To summarise and as depicted in Table 2, larger runway margins, better runway surface conditions, a more aggressive use of the brakes and a pitch closer to the neutral reduces the probability of the critical landing. In the opposite direction, longer and faster touchdowns increase this probability.

Note that the ‘Brake Application Profile’ (i.e. the brake pedals mean application between the aircraft wheel spin and the landing end) is the most relevant contributor. To increase this variable, the flight crew can increase the maximum brake pedal deflection used during the landing run or to reach this deflection earlier, maintaining it. The maximum brake application used to estimate the ULD would be represented by a mean of 100%, while the median of the analysed fleet is circa of 47% as shown in Fig. 13.

To evaluate the regression model obtained, the value of the adjusted R ² (R ² _adj) is used. This criterion is a statistical measure of how close the data are to the fitted regression line, varying between 0 (the output cannot be explained by the inputs) and 1 (the output is perfectly explained by the inputs). In general, the higher the R ², the better the model fits your data. The RsqGLM R function from package modEvA was used to estimate the R ² _adj of the current model [Reference Barbosa, Real, Munoz and Brown43], resulting in a R ² _adj of 0.2525447 in accordance with Pearson method or 0.3434947 with the Nagelkerke one.

The model comprises different types of predictor variables. The runway margins and the runway surface condition are environmental conditions from which the flight crew has a limited or even no control. The groundspeed and the pitch on ground are available and directly manageable parameters. The touchdown point and the brake application profile are also controllable, but they are a combination of other inputs from the pilots. The touchdown point can be described in terms of the height, the Δv_ref and the TLA at the runway threshold. The flare procedure and its consequent descent rate decreases the touchdown point, whose influence is mainly given by the time between the maximum pitch and the wheel spin. The brake application profile comprises the moment that the flight crew starts the brake pedal application, the moment that they reach the maximum value and the maximum pedal value by itself. Table 3 presents the variables that the flight crew is able to handle.

Table 3. Pilot handling impact over the predictor variables

¹The height is already under the standard one

4.4 Variability management

The model emphasises the impact of the selected parameters and, consequently, recognises which points of the operation should be preferably managed to reduce the RO likelihood of the analysed fleet.

It is recommended, for the sample fleet, a decrease of the aircraft speed during the approach and landing phase as well as an increase of the maximum brake pedal deflection in order to reduce the likelihood of an overrun. On other hand, the height at the runway threshold is under the recommended one. But this variability is improving the current system output and does not need to be addressed, at least because of ROs.

After identifying the points of attention of the fleet under analyses, they may be addressed in the flight crew recurring training, or by procedure reinforcement in safety meetings, or even by changes in the standard operating procedures.