Impact Statement
Recent advances in the development of small-scale autonomous vehicles require energy-efficient path planning strategies to enable navigation in highly unsteady flows. In this work, we investigate navigation in unsteady fluid flows, specifically how three-dimensional flow structures can be leveraged for effective navigation with limited temporal information of the flowfield. We study energy-efficient trajectory planning in 3D wake flows. This work shows that 3D vortical structures can be leveraged for energy-efficient navigation, and that their trajectories differ from the case of 2D wake flows. We identify successful navigation strategies using limited temporal prediction capabilities, and hence have the potential to develop sensor-based autonomous vehicle navigation strategies in complex 3D unsteady flows.
1. Introduction
Small-scale aerial and marine vehicles are gaining traction in applications typically considered high risk for manned vehicles. For these operations, vehicles need to traverse unsteady flows, such as flying in urban environments (Watkins et al., Reference Watkins, Burry, Mohamed, Marino, Prudden, Fisher, Kloet, Jakobi and Clothier2020), in the wake of artificial and natural structures and navigating behind large marine vehicles for maritime operations (Shukla & Sinha 2015). Such wake flows are associated with strong unsteady three-dimensional (3-D) vortical disturbances. Autonomous vehicles often experience adverse effects when encountering such flows without prior information (Zereik et al., Reference Zereik, Bibuli, Misković, Ridao and Pascoal2018).
There have been studies on the effect of wake flows on the stability and performance of biological flyers and swimmers. The effect of 3-D vortical perturbations generated by unsteady bluff-body wakes on the aerodynamic coefficients and moments of insect flight and hummingbird flight show variable responses depending on the orientation and relative strength of the vortical perturbations (Ortega-Jimenez et al., Reference Ortega-Jimenez, Sapir, Wolf, Variano and Dudley2014; Shyy et al., Reference Shyy, Kang, Chirarattananon, Ravi and Liu2016). The effect of three-dimensionality on locomotion is also studied in fish swimming (Lauder Reference Lauder2015; Liao Reference Liao2007; Maia et al., Reference Maia, Sheltzer and Tytell2015) where fish undulate relative to the vortical structures for energy-efficient navigation. These studies have been conducted at high speeds and also consider the interaction between the swimmer/flyer with the flowfield. Understanding how the 3-D flow structures can be leveraged for navigation offers insights into biological locomotion and the development of autonomous navigation strategies. Hence, we consider trajectory planning for laminar 3-D wakes.
Over recent years, there has been extensive research on trajectory-planning strategies for the navigation of autonomous vehicles in complex background flows. Given complete knowledge of the underlying environment, optimisation methods such as graph-based algorithms (Kularatne et al., Reference Kularatne, Bhattacharya and Hsieh2016), and stochastic optimisation (Subramani & Lermusiax Reference Subramani and Lermusiaux2016) have been used to obtain optimal trajectories. Recently, reinforcement learning has been utilised for navigation in unsteady flows using limited local information (Colabrese et al., Reference Colabrese, Gustavsson, Celani and Biferale2017; Gunnarson et al., Reference Gunnarson, Mandralis, Novati, Koumoutsakos and Dabiri2021; Jiao et al., Reference Jiao, Ling, Heydari, Heess, Merel and Kanso2021). Additionally, the Lagrangian coherent structure (LCS) theory has been used for energy-efficient path planning (Senatore & Ross Reference Senatore and Ross2008). Krishna et al., Reference Krishna, Song and Brunton2022 proposed a finite-horizon model-predictive control approach to identify energy-efficient trajectories and their connection to LCSs for navigating unsteady flows.
In this study, we investigate the effect of three-dimensionality on navigation in unsteady wakes using the finite-horizon model-predictive control approach used in Krishna et al., Reference Krishna, Song and Brunton2022. We consider underactuated point swimmers navigating in the 3-D wake of a cylinder at a Reynolds number of 300, where the flow exhibits three-dimensionality due to secondary instabilities, and compare their performance when navigating in 2-D wakes. We reveal that secondary vortices in 3-D flow facilitate faster navigation with limited information. The paper is outlined as follows. The computational set-up and the flow physics of the cylinder flow are introduced in section 2. The finite-horizon model-predictive control approach is discussed in section 3. Section 4 provides the results for trajectory planning in cylinder wakes. The conclusions are given in section 5.
2. Computational set-up
We consider a 3-D incompressible wake behind a circular cylinder obtained from a direct numerical simulation (DNS) at a diameter (D) based on a Reynolds number, Re = U ∞ D/ν of 300, where U ∞ and ν are free-stream velocity and kinematic viscosity, respectively. The simulation is performed using the incompressible flow solver Cliff, (Cascade Technologies Inc.) based on a second-order accurate finite volume method for spatial discretisation and a fractional-step method for time stepping (Ham & Iaccarino Reference Ham and Iaccarino2004; Ham et al., Reference Ham, Mattson and Iaccarino2006). The computational domain extends to − 20 ≤ x/D ≤ 30 in the streamwise direction. The transverse extent is − 40 ≤ y/D ≤ 40 and the spanwise extent is 0 ≤ z/D ≤ 4 to capture the wavenumber of secondary instabilities with a uniform discretisation of 80 grid cells and a time step of Δt = 0.005. Hybrid grids are used with a structured mesh close to the cylinder and an unstructured mesh in the far-field region. We also obtain a 2-D flow over a cylinder at the same Reynolds number using the same domain in the x, y directions. It amounts to approximately 0.1 and 8.3 million cells for 2-D and 3-D wakes, respectively. Further details on the computational set-up can be found in Kim et al., Reference Kim, Godavarthi, Rolandi, Klamo and Taira2024. For finite-horizon trajectory planning and visualisation, we choose a subdomain (x/D, y/D, z/D) ∈ [−2,10] × [−2,2] × [0, 2].
3. Finite-horizon model-predictive control
We use a finite-horizon model-predictive control (MPC) approach developed by Krishna et al., Reference Krishna, Song and Brunton2022 to perform trajectory optimisation of a point swimmer. The swimmer dynamics is modelled as
\begin{align} \dot {\boldsymbol {x}}(t) = \boldsymbol {v}(\boldsymbol {x}(t),t)+\boldsymbol {u}(t), \end{align}
where x , u ∈ ℝ n are the position vector and actuation velocity of the swimmer while v is the background flow velocity and n = 2, 3 for 2-D and 3-D flows. The background flow velocity is obtained from the DNS of the cylinder wake. When the actuation velocity is zero, the swimmer acts as a passive drifter. The swimmer dynamics is numerically integrated using a time step of Δt = 0.05. The actuation velocity is determined from the MPC optimisation of the cost function given by
\begin{align} J = \int \limits _{t_0}^{t_0+T_H}[\boldsymbol {e}(\tau )^T\mathbf {Q}\boldsymbol {e}(\tau )+\boldsymbol {u}(\tau )^T\mathbf {R}\boldsymbol {u}(\tau )]d\tau ,\end{align}
subject to constraints on the component-wise actuation velocity with |u
i
(t)| ≤ η
i
, where η
i
is the actuation velocity bounds on the ith velocity component. Here, T
H
is the time horizon over which the cost function is minimised, and
e
is the error of the current state from a target state,
x
(t),
e
(t) =
x
(t) −
x
target. The matrix
$\mathbf {Q}\in \mathbb {R}^{n\times n}$
is positive semi-definite and penalises the state error throughout the trajectory, and
$\mathbf {R}\in \mathbb {R}^{n\times n}$
is a positive definite matrix penalising the actuation effort. Here, the actuation velocity of the swimmer is penalised, although the acceleration could also be optimised.
For the current problem,
$\mathbf {Q}$
is set to the identity matrix. When the actuation velocity is large i.e. |u| ≥ U
∞ (free stream), the trajectory reaches the target directly. We consider underactuated scenarios to see how the swimmer exploits the background flow. The local background velocity for the cylinder wake varies in the streamwise, transverse and spanwise (x, y, z) directions, with the streamwise velocity being dominant. Thus, the actuation efforts are penalised differently in different directions, and
$\mathbf {R}$
is a diagonal matrix where the ith diagonal element is given as R
ii
= γ
i
and γ
i
is the actuation penalisation in the ith direction. The time horizon T
H
is another key parameter, as larger T
H
requires larger predictive capabilities i.e. more temporal information about the background flow to the swimmer.
4. Navigating three-dimensional wake flows
We consider trajectory planning for wake crossing in 2-D and 3-D cylinder wakes at Re = 300 as a canonical problem to examine the effect of three-dimensionality on swimmer navigation. At this Re, the secondary instabilities (referred to as modes A and B) result in three-dimensionality (Williamson Reference Williamson1996). Modes A and B are developed in the vortex cores (seen as less coherent spanwise vortices due to vortex tilting (Aleksyuk & Heil Reference Aleksyuk and Heil2023)) and the braid regions between consecutive spanwise vortices (seen as streamwise-elongated vortices in figure 1(a)), respectively. These secondary instabilities result in the aperiodic nature of wake flow with a dominant frequency of St = 0.202, contrary to the periodic flow at a frequency St = 0.212 developed over a 2-D cylinder at the same Re.
Figure 1. Trajectories of swimmers crossing the wake for 3-D flow over a cylinder at Re = 300 visualised using an iso-surface of Q-criterion Q = 0.5, coloured by spanwise vorticity ω z in (a) isometric view. (b) Probability distribution of total navigation time in 2-D and 3-D wakes.
We consider wake-crossing scenarios (around 60 samples). These trajectories are shown in red in figure 1(a). The start (filled circles) and target transverse y locations are on either side of the cylinder wake (y < 0 for start and y > 0 for target positions) on the same spanwise plane. The streamwise x locations are chosen randomly in the cylinder wake. The background flow is initialised at the same phase of vortex shedding for both 2-D and 3-D flows. We consider five different initial phases of vortex shedding as the starting background flow for all the samples. Since the optimisation function is performed for each component, the penalisation for actuation velocity is also performed component-wise. For these wake-crossing scenarios, the swimmer is sensitive to actuation in a transverse direction, i.e. when not penalised in the transverse direction, the swimmer reaches the target in a straight line, as discussed later. Here, we present the underactuated cases with actuation bounds on the velocity as 0.9, 0.2, 0.5 in the x, y, z directions with R
ii
using (γ
1, γ
2, γ
3) = (0.1,10,0.1). The time horizon for all cases is chosen as T
H
= 10Δt ≈ 0.1T
p
, where T
p
is the dominant wake-shedding period. The time horizon can also be measured relative to the size of the vortical structures in the flowfield. The wavelength of the mode B (Λ
B
) instability, resulting in the braid-like region, is Λ
B
= 0.8D (Barkley & Henderson Reference Barkley and Henderson1996), hence this time horizon translates to 0.5 convective time units based on the diameter of the cylinder and 0.625 time units relative to the size of the braid-like structures. This indicates that flowfield information of approximately one half of the size of these vertical structures is enough for successful navigation. The obtained trajectories for T
H
≥ 0.1T
p
indicate that the swimmers can reach the target when the time horizon is approximately at least one-tenth of a period. For T
H
= 10, the same trends are observed for 0.85 ≤ η
1 ≤ 0.95, 0.1 ≤ η
2 ≤ 0.3, 0.1 ≤ η
2 ≤ 0.5. An instance of the effect of spanwise actuation bounds on the swimmer trajectory is discussed in the Appendix (depicted in figure 4). Since the optimisation strategy minimises ||e(t)|| in (3.2), we consider the swimmer to be successful when ||e(t)|| ≤
$\epsilon$
D, which in this study is set to
$\epsilon$
= 1/3.
To compare the navigation performance in 2-D and 3-D wakes, we use the navigation time N T , the time taken by the swimmer to reach the target, as a performance metric. The probability distribution of N T for the sampled scenarios averaged over the three different initial background flow conditions is visualised in figure 1(b). We observe that distribution is shifted toward lower N T for 3-D wake navigation identifying faster navigation in 3-D wakes. The probability distribution also shows cases when N T → ∞ for unreachable scenarios, Thus the swimmers navigating in a 3-D wake are more likely to reach the target position while being faster than those navigating in a 2-D wake.
Figure 2. An xy-view of wake-crossing trajectories in 3-D (top row) and 2-D (bottom row) wakes visualised using |ω z| for when (a) 3-D navigation is faster (blue), (b) navigation in 2-D cannot reach the target (black) and (c) navigation times in 2-D and 3-D wakes are similar (red).
We divide the obtained trajectories into three scenarios as depicted in figure 2: (i) when the 3-D navigation is faster than the 2-D one (blue in figure 2a), (ii) when swimmers cannot reach the target (black in figure 2b) and (iii) when navigation in two dimensions is faster than in a 3-D wake (red in figure 2c). For (iii), the navigation time difference is approximately 20% of the wake-shedding period (−ΔN T <0.2T p ). The main difference between (i) and (iii) is the initial traverse locations y 0 (filled circles): for (i) they are outside of the wake whereas for (iii) the start locations are in the wake region, resulting in lower background velocity from the start. From figure 2(a), the underactuated swimmer traverses significantly in the streamwise direction before reaching the target, whereas in figure 2(c) we observe that, due to the current actuation parameters and the initial and target locations of the swimmer relative to the cylinder wake, the swimmer can directly cross the wake (almost in a straight line). This shows that streamwise navigation is faster in 3-D wakes.
Figure 2(b) shows such cases where the swimmer fails to reach the target location in a 2-D wake (black) but succeeds in a 3-D wake (blue). The transverse target locations y target > 1.4D (unfilled circles) are farther from the wake, located in the free stream, for these cases. The 2-D navigation with a finite horizon fails whereas the 3-D wake navigation succeeds. Although the target locations are in the free stream, the secondary vortices in the braid regions in a 3-D wake provide low-velocity regions, allowing the target to be reached and expanding the reachability boundaries. Even in a 3-D wake, there are locations too far from the wake where the swimmer fails to reach the target for a given time horizon (shown by the leftmost trajectory in black in figure 2b (top)).
Figure 3. Trajectory optimisation for a wake-crossing scenario in (a) 3-D and (b) 2-D wakes visualised using |ω z|. (c–d) Instantaneous trajectories during the navigation in 3-D (top) and 2-D (bottom) wakes. (e–f) Zoomed-in view of the evolution swimmer trajectory and background flow visualised using Q = 0.1 and coloured using ω y in a 3-D wake.
We now investigate the flow physics that the swimmer leverages for faster navigation in 3-D wakes. We consider a scenario where 3-D wake navigation is faster (ΔN T > 0), as depicted in figure 3(a-b) with an initial location of x 0 = (4.34,−2.65,1.0) (denoted by filled circle) and the target position of x target = (3.36,0.29,1.0) (denoted by unfilled circle) and ΔN T = 8.3 ≈ 1.8T p . In both 2-D and 3-D wake navigation, the swimmer initially travels downstream due to the lower actuation velocity of the swimmer compared with the background flow. A few time instants for the downstream navigation are shown in figure 3(c). We observe that the swimmer in a 3-D wake ‘redirects’ towards the goal earlier than that in a 2-D wake, as seen at t = 17.5, even though both the swimmers reach similar y locations. The swimmer in a 2-D wake travels farther downstream before it can redirect towards the target. Once the y location of the swimmer is in the cylinder wake, utilising the low background velocity the swimmer redirects and travels upstream in the wake towards the target, as shown in figure 3(d). The swimmer trajectory oscillates in the x and y directions relative to the spanwise vortices in the cylinder wake, with fewer oscillations observed in a 3-D wake compared with a 2-D wake. The upstream navigation trajectory in the 2-D wake in figure 3(b) is similar to the ones identified by Gunnarson et al. (Reference Gunnarson, Mandralis, Novati, Koumoutsakos and Dabiri2021), where the swimmer leverages the induced velocities in the x and y directions by the spanwise vortices.
The zoomed-in view of the time instants when the swimmer in a 3-D wake redirects towards the target (green box in figure 3a) and the time instants of the swimmer navigating upstream (yellow box in figure 3b) are shown in figure 3(e–f). The instantaneous swimmer location (black dot) and its trajectory (grey) are shown relative to a transverse vortex (shown using ω y ) as it induces velocity in x and z directions. Figure 3(e) depicts the swimmer navigation under the influence of a counter-clockwise vortex. From, t = 15.5 to t = 17, the swimmer first travels in the negative z direction (t = 16, 16.5) and then travels in the upstream (negative x) direction, (t = 17) following the induced velocity and the background flow, thereby redirecting toward the upstream target earlier than its 2-D counterpart.
For efficient upstream navigation in the 3-D wakes, two mechanisms are in play: (i) the vorticity in the transverse and streamwise directions and (ii) the reduced coherence of spanwise vortices. Figure 3(f) depicts the swimmer navigation under the influence of a clockwise transverse vortex while travelling upstream in the wake. From t = 23.25 to t = 24, the swimmer first travels in the positive z direction and then in the negative x direction (t = 23.5, 23.75), leveraging the local orientation of background flow caused by the transverse vortex. This shows that the swimmer in a 3-D wake travels in positive or negative z direction to effectively leverage the transverse vortices for upstream travel. The swimmer travels in the spanwise direction according to the orientation of subdominant transverse and streamwise vortices. These secondary vortices provide low-velocity regions and change the background flow orientation due to the induced velocity. The swimmer leverages these vortices through the spanwise motion to effectively ‘redirect’ toward the target for faster upstream navigation.
5. Conclusions
We investigated the influence of three-dimensionality on the navigation of underactuated swimmers in unsteady wakes using finite-horizon MPC for trajectory optimisation. We compared the obtained trajectories for navigation in 2-D and 3-D cylinder wakes at Re = 300. For both scenarios, the time horizon needed to reach the target is only one-tenth of the wake-shedding period. This makes trajectory optimisation for bluff-body wake navigation sensor friendly. We also identified that the swimmer can navigate faster in 3-D wakes compared with 2-D wakes for most scenarios. Through spanwise motion, the swimmer can effectively leverage the secondary vortices, specifically transverse vortices to redirect towards the target faster. The low coherence of spanwise vortices and the presence of secondary vortices also facilitate faster upstream navigation when compared with 2-D wake navigation.
Acknowledgements
We thank Y. Kim for providing the flowfield data and K. Fukami for his valuable insights.
Data availability
The code used in this study is openly available at https://github.com/karkris41295/single-agent-MPC-FTLE. The data that support the findings of this study are available from the author upon reasonable request.
Author contributions
V.G., K.K. and K.T. conceived the idea. V.G. and K.K. performed computations. All authors contributed to the writing of the manuscript and its revision. S.L.B. and K.T. secured funding.
Funding
We acknowledge funding from the US Air Force Office of Scientific Research (FA9550-21-1-0178). K.T. thanks the support from the US Department of Defense Vannevar Bush Faculty Fellowship (N00014-22-1-2798).
Competing interests
The authors declare no conflict of interest.
Ethical standards
The research meets all ethical guidelines, including adherence to the legal requirements of the study country.
Appendix
The underactuated swimmer trajectories shown in the main manuscript are obtained using the actuation bounds of (η 1,η 2,η 3) = (0.9,0.2,0.5) in the x, y and z directions respectively. While the lower actuation bound in the y direction is needed to avoid crossing the wake in a straight line, we here address the effect of actuation bounds in the spanwise direction. Let us consider the scenario discussed in figure 3. Figure 4 shows the trajectory evolution of the swimmer when η 3 = 0.2 (red) and η 3 = 0.5 (blue). Here, we use the same actuation bounds of (η 1,η 2) = (0.9,0.2) in x and y directions. We observe that both trajectories are similar irrespective of the difference in the actuation bound in the z direction. Since the secondary vortices are weaker than the spanwise vortices, the induced velocity by the secondary vortices in the z direction is also quite small, resulting in less actuation requirement in the z direction.
Figure 4. Trajectory evolution for a wake-crossing scenario in a 3-D wake in the three directions for different actuation bounds in z-direction (η 3).
Open access
