Resolution

In most cases, transmitting and receiving antennas are deployed so as to form a virtual uniform linear array. In this case, the angular resolution for a forward-looking radar is:

(1)$$\Delta \psi_{MIMO} = {\lambda\over 2 \cos{( \psi) }} {1\over N_{ch}d_x}$$

where λ is the carrier wavelength, N _ch = N _Tx × N _Rx is the number of available channels, d _x is the spacing between nearby elements in the virtual array (assuming the virtual array is represented as a collection of monostatic elements), and ψ is the off-boresight angle, as shown in Fig. 1. It is worth noting that in most cases the antenna layout is such that d _x = λ/4 to ensure non-ambiguous imaging, after which one has that angular resolution is fully determined by the geometrical factor cos ψ and the number of elements N _ch. For any value of d _x, the product N _ch d _x represents the length of the virtual array.

Figure 1. Angular resolution. Black dashed curves: for a front-looking MIMO radar employing 32, 64, and 128 channels. Continuous curves angular resolution for a SAR employing a synthetic aperture of 0.3, 0.7, and 1.2.

In the case of SAR, instead, the resolution is achieved by processing a set of data acquired as the vehicle travels a certain distance A _s, called the synthetic aperture, which can be thought of as the length of an equivalent (monostatic) virtual array deployed longitudinally (i.e. in the direction of motion, orthogonal with respect to the MIMO array). Accordingly, the angular resolution for SAR imaging is obtained as:

(2)$$\Delta \psi_{SAR} = {\lambda\over 2 \sin{( \psi) }} {1\over A_s}$$

where the geometrical factor is now sin(ψ) due to the longitudinal deployment. Comparing (1) with (2), and by Fig. 1, it is immediate to see that SAR imaging can serve as a natural complement to conventional radar in the automotive context. SAR imaging does not bring any improvement for targets right at boresight. Yet, it does provide a far superior resolution for targets slightly off-boresight. For this reason, in the automotive context SAR imaging has primarily been considered to produce high-resolution mapping of the environment to the sides of the road, suggesting SAR techniques could be most suited to urban scenarios [Reference Stanko, Palm, Sommer, Kloppel, Caris and Pohl8–Reference Feger, Haderer and Stelzer10].

SNR

The SNR computation for radar and SAR imaging can be approached by simply assuming some processing scheme where consecutive pulses are coherently integrated over some coherent integration time T _c, which is proportional to the SNR improvement factor over single-pulse data per processed channel. The effective coherent integration time depends largely on which processing scheme is adopted. Any accurate SAR imaging algorithm accounts for any effect related to the variation of the sensor-to-target distance across different pulses. This includes the cases where the range shift between signals acquired at different times exceeds one range bin, referred to as range migration, and where the phase variation across different pulses cannot be correctly modeled by a linear law. In this way, under the assumption that range migration and phase variations are correctly matched, the coherent integration time is only bounded by the physical antenna beam width, as it is the case for space-borne and air-borne SARs. On the other hand, radar imaging is (typically) implemented under the assumptions that range migration can be neglected and phase variations are linear. Such assumptions allow for a significant simplification of the signal model and all related processing algorithms. Yet they set an upper bound for the coherent integration time. As for range migration, the requirement is that the range shift over time does not exceed range resolution:

(3)$$\Delta R = v_{ego} \cos{( \psi) } T_c < 2\delta_r$$

where v _ego is the vehicle forward velocity, δ _r is range resolution, and the factor 2 is used to allow for some smoothness of the waveform spectrum. Notice that this bound is more stringent at the boresight of the MIMO radar (ψ ≈ 0).

The non-linear nature of the phase history can be accounted for by bounding the residual parabolic component. Let's suppose the vehicle moves at a velocity v _ego on a rectilinear path. It is straightforward to develop in Taylor series around a point τ ₀ the hyperbolic law of distances between the sensor and a generic target:

(4)$$\eqalign{R( \tau) & = \sqrt{R_0^2 + ( v_{ego} \tau) ^2} \cr & \approx {\underbrace{R}_{\text{constant}}} + \underbrace{\cos( \psi) v_{ego} \Delta \tau}_{\text{linear}} + \ \underbrace{{v_{ego}^2 \sin^2( \psi) \over 2R}\Delta \tau^2}_{\text{\,parabolic}} }$$

where R is the distance between the radar and the target at τ = τ ₀ and τ is the slow-time. All the quantities just described are also depicted in Fig. 2. We now upper bound the phase parabolic component at π/2 obtaining:

(5)$$\Delta \varphi = {4\pi\over \lambda}{v_{ego}^2 \sin^2( \psi) \over 2R}\left({T_c\over 2}\right)^2 < {\pi\over 2}$$

To quantify the limits on T _c , we assume here a set of parameters representative for high-resolution urban mapping, such as f ₀ = 77 GHz, v _ego = 10 m/s, δ _r = 0.15 m, and R ₀ = 10 m. Plugging those values into (3) and (5) one gets that the coherent integration time is on the order of 30 ms for all targets within ±60 deg off-boresight. This value sets a physical limit to the SNR improvement factor achieved by integrating over time, unless SAR processing is adopted.

Figure 2. Geometry of the problem.

Of course, in conventional radar imaging both SNR and angular resolution can always be improved by increasing the number of channels N _ch, that is by moving toward more sophisticated and expensive devices. On the other hand, the present analysis has just shown that SAR processing allows for large performance improvements by exploiting the natural motion of the ego-vehicle, therefore opening the way to the concept of high-resolution mapping using low-cost devices. The price to pay is obviously an increased complexity and computational burden, as we discuss in the next section.

The 3D2D processing scheme

In this section, we described another possible approach to achieve real-time SAR imaging. This approach will be called from now on 3D2D and the reason for this name will be clear at the end of the section. While both algorithms (FFBP and 3D2D) are suited for a future real-world implementation, we deem that the 3D2D scheme is possibly simpler to implement, provides high-quality images, and is generally faster. The block diagram of the 3D2D processor is depicted in Fig. 3.

Figure 3. 3D2D block diagram.

As the FFBP, the 3D2D starts from a set of coregistered and bandpass low-resolution images. These images are obtained with a simple TDBP over all the virtual channels and for each slow-time instant. We remark that, by focusing each image on the same common grid, we compensate for range migration and we take into account also phase curvature. This is significantly different with respect to the so-called Doppler beam sharpening, which does not consider both these aspects. Another input data are the vehicle's trajectory from the Navigation Unit (NU). The way in which these data will be used will be described later on in this section.

The 3D2D workflow proceeds as follows:

1. (i) Each low-res image is demodulated into baseband using a linear law of distances:

   (6)$$I_{bb}( r,\; \psi,\; \tau_n) \approx I_{MIMO}( r,\; \psi,\; \tau_n) e^{-j\,{4\pi\over \lambda}R( r, \psi, \tau_n) }$$
   The approximation is used in place of the equality just to remember that the data are placed into baseband with an approximated law of distances given by:
   (7)$$R( r,\; \psi,\; \tau_n) = R_0( r,\; \psi) + v_r( \psi) ( \tau_n - \tau_0) $$
   where R ₀(r,  ψ) represents the distance from the center of the aperture to each pixel, τ ₀ is the slow-time of the center of the aperture, and v _r(ψ) is the radial velocity of the car with respect to a fixed target in (r,  ψ). This velocity is calculated exploiting the nominal trajectory provided by the NU at the center of the aperture.

   The linear approximation of the true hyperbolic range equation holds for short aperture and/or far-range targets as already expressed in (5).

2. (ii) The Fourier transform (FT) in the slow-time dimension of the baseband data is taken:

   (8)$$I\left(r,\; \psi,\; f_d = -{2\over \lambda}v_r \right) = {\cal F}\{ I_{bb}( r,\; \psi,\; \tau_n) \} $$
   where v _r is the radial velocity and ${\cal F}$ is the Fourier transform in the slow-time direction. The transformed dataset is now in range, angle, and Doppler frequency domain which can be easily converted in range, angle, radial velocity. The output is the 3D Range–Angle–Velocity (RAV) data cube. The number of frequency points over which the FT is computed is an arbitrary parameter and can be chosen to provide a very fine sampling in the radial velocity domain. We remark that the FT simply add a linear phase term to the data and sum the result. This coherent summation is what generates the SAR image which now should just be extracted from the RAV cube.

3. (iii) The extraction of the SAR image is performed by interpolating the 3D cube over a 2D surface. The $( r_{fine},\; \, \psi _{fine},\; \, v_r^{NAV})$ coordinates of the 2D surface are calculated considering the high-resolution nature of the final SAR image and the nominal radial velocity provided by the vehicle's navigation unit (eventually corrected with the estimated residual velocities from the autofocus procedure):

   (9)$$\matrix{& I_{SAR}( r_{\,fine},\; \psi_{\,fine}) \cr & \quad = I( r \rightarrow r_{\,fine},\; \psi \rightarrow \psi_{\,fine} ,\; v_r \rightarrow v_r^{NAV}) }$$
   where → is the interpolation operation, r _fine and ψ _fine are the range and angle coordinates of the final SAR image, and $v_r^{NAV}$ is the radial velocity of each pixel in the fine resolution grid as calculated by navigational data and corrected by the autofocus procedure.

The interpolation in (9) has a noticeable geometrical meaning. It describes the formation of an SAR image in terms of the intersection of the curved surface $v_r^{NAV}$ with the 3D RAV data cube. In Fig. 4, some Doppler (or velocity) layers of the RAV cube are represented as horizontal light blue panels. The maximum of the absolute value is usually taken in the Doppler domain to detect targets. The detected maximum is provided at the bottom of the layers in Fig. 4.

Figure 4. A visual interpretation of the 3D2D algorithm. In light blue, the velocity (or Doppler) layers of the Range–Angle–Velocity (RAV) data cube. The maximum of the absolute value in the velocity domain is usually taken and it is displayed at the base of the RAV cube. The SAR image is embedded in the RAV cube and can be extracted with a simple interpolation. The curved surface represents the SAR image.

The curved surface extracted from the RAV cube and representing the SAR image is also depicted in Fig. 4. We remark that the curvature of that surface depends on the vehicle's motion and it is provided by the navigation unit (and refined by the autofocus procedure). A final note should be done about autofocus. High frequencies call for high accuracy in the knowledge of the vehicle's motion during the synthetic aperture time. At the wavelength used in the automotive industry, however, no navigation unit can provide sufficient accuracy, therefore a data-drive residual motion compensation must be employed. In our case, we used the technique already detailed in [Reference Manzoni, Tagliaferri, Rizzi, Tebaldini, Guarnieri, Prati, Nicoli, Russo, Duque, Mazzucco and Spagnolini11] that is based on the estimation of residual Doppler on a set of detected ground control points sparsed in the imaged scene.

Computational cost

To prove the real-time capabilities of the proposed algorithm, we can now derive the rough number of operations (RNO) required to focus an SAR image. This number will provide an estimate of the computational burden needed to focus a single image using the 3D2D algorithm. We remark once again that this is just a rough order of magnitude and it is by no mean a precise number of operations which is instead inherently dependent upon the hardware and software implementation.

The processing starts with the so-called range compression of the raw radar data. In any frequency-modulated continuous-wave radar, this can be done with a simple fast Fourier transform (FFT) [Reference Meta, Hoogeboom and Ligthart14]. This step requires a total number of operations equal to:

(10)$$N_f \log_2N_f\times N_{ch} \times N_{\tau}$$

where the first term represents the number of operations required to perform an FFT over N _f frequency samples [Reference Cooley and Tukey15], while the last two terms highlight that these operations must be repeated for each of the N _ch channels and all the N _τ slow-time samples.

The next step involves the TDBP of the data to form the stack of low-resolution MIMO images. For each pixel in the BP grid, we need to:

1. (i) Calculate the bistatic distances from each real antenna phase center to the considered pixel. This operation is composed of six squares and two square roots (Pythagorean theorem) for a total of eight operations per channel.

2. (ii) Interpolate the range compressed data using a linear or nearest neighbor interpolator accounting for two operations per channel.

3. (iii) Modulate the signal, which is just a single complex multiplication per pixel.

4. (iv) Accumulate the back-projected signal for each channel with a single complex summation.

We remark that these operations must be computed for each pixel in the BP grid and for each slow-time; thus, the final number of operations is equal to:

(11)$$N_r \times N_a \times N_\tau \times 12N_{ch}$$

where N _r and N _a are the sizes of the backprojection grid in the range and angular direction, respectively.

Now the core of the 3D2D algorithm starts with a demodulation of each of the $N_\tau$ low-resolution images. For each pixel, a single multiplication is performed to compute the distances in (7) and a multiplication is performed to demodulate the data. This consideration leads to a number of operations equal to:

(12)$$N_\tau \times N_a \times N_r \times 2$$

The next step is a slow-time FFT to be computed over each pixel, leading to:

(13)$$N_a \times N_r \times N_v\log_2 N_v$$

where N _v is the number of Doppler frequency (or velocity) points computed by the FFT, a typical value is $N_v = 8N_\tau$.

The final step is the 3D interpolation. For each pixel of the output fine resolution polar grid, we have to compute the radial velocity (see (9)). This step requires a number of operations equal to the size of the output grid N _r × N _ah where N _ah is the size in the angular direction of the fine resolution polar grid.

Now, the interpolation takes place with a number of operations in the order of 27 × N _r × N _ah. The number 27 is derived from the fact that each pixel in the output grid is derived by combining the 27 closest neighbors, i.e. the 3 × 3 × 3 cube centered on the pixel.

These considerations lead to a total number of operations equal to:

(14)$$\matrix{& \underbrace{N_\tau \times N_a \times N_r \times 2}_{\text{demodulation}} + \underbrace{N_a \times N_r \times N_v\log_2 N_v}_{\text{FFT}}\cr & \quad + \underbrace{28 \times N_r \times N_{ah}}_{\text{interpolation}} }$$

Typical values of all the parameters involved in the computation of the RNO are listed in Table 1. Plugging these values into (10), (11), and (14) we reach an RNO of about ~329 millions. For comparison, a 9-year-old iPhone®5S has a raw processing power of 76 GigaFlops [Reference Victor16], therefore it could, in principle, focus an SAR image in 6 ms. We deem that, with dedicated hardware implementations like on GPU/FPGA, it is easily possible to achieve real-time imaging of the surroundings.

Table 1. Typical values of all the parameters involved in the computation of the rough number of operations (RNO)