Covers differentiation and integration, higher derivatives, partial derivatives, series expansion, integral transforms, convolution integrals, Laplace transforms, linear and time-invariant systems, linear ordinary differential equations, periodic functions, Fourier series and transforms, and matrix algebra.

This chapter presents mathematical details relating to the Fourier transform (FT), Fourier series, and their inverses. These details were omitted in the preceding chapters in order to enable the reader to focus on the engineering side. The material reviewed in this chapter is fundamental and of lasting value, even though from the engineer’s viewpoint the importance may not manifest in day-to-day applications of Fourier representations. First the chapter discusses the discrete-time case, wherein two types of Fourier transform are distinguished, namely, $\ell_1$-FT and $\ell_2$-FT. A similar distinction between L1-FT and L2-FT for the continuous-time case is made next. When such FTs do not exist, it is still possible for a Fourier transform (or inverse) to exist in the sense of the so-called Cauchy principal value or improper Riemann integral, as explained. A detailed discussion on the pointwise convergence of the Fourier series representation is then given, wherein a number of sufficient conditions for such convergence are presented. This involves concepts such as bounded variation, one-sided derivatives, and so on. Detailed discussions of these concepts, along with several illuminating examples, are presented. The discussion is also extended to the case of the Fourier integral.

Chapter 6 starts out with a physics motivation, as well as a mathematical statement of the problems that will be tackled in later sections. First, polynomial interpolation is carried out using both the monomial basis and the Lagrange-interpolation formalism, sped up via the barycentric formula. This includes a derivation of the error and an emphasis on using unequally spaced points (Chebyshev nodes). Second, cubic-spline interpolation is introduced. Third, a section is dedicated to trigonometric interpolation, carefully working through the conventions and formalism needed to implement one of the most successful algorithms ever, the fast Fourier transform (FFT). Fourth, the topic of linear least-squares fitting is tackled, including the general formalism of the normal equations. The second edition includes a substantive new section on statistical inference, covering both frequentist and Bayesian approaches to linear regression. Nonlinear least-squares fitting is covered next, including the Gauss-Newton method and artificial neural networks. The chapter is rounded out by a physics project, on the experimental verification of the Stefan-Boltzmann law, and a problem set. In addition to providing a historical background on black-body radiation, the physics project shows an example of nonlinear least-squares fitting.

Chapter 6: In this chapter, we explore the role of orthonormal (orthogonal and normalized) vectors in an inner-product space. Matrix representations of linear transformations with respect to orthonormal bases are of particular importance. They are associated with the notion of an adjoint transformation. We give a brief introduction to Fourier series that highlights the orthogonality properties of sine and cosine functions. In the final section of the chapter, we discuss orthogonal polynomials and the remarkable numerical integration rules associated with them.

The Fourier series is introduced as a very useful way to represent any periodic signal using a sum of sinusoidal (“pure”) signals. A display of the amplitudes of each sinusoid as a function of the frequency of that sinusoid is a spectrum and allows analysis in the frequency domain. Each sinusoidal signal of such a complex signal is referred to as a partial, and all those except for the lowest-frequency term are referred to as overtones. For periodic signals, the frequencies of the sinusoids will be integer multiples of the lowest frequency; that is, they are harmonics. Pitch is a perceived quantity related to frequency, and it may have a complicated relationship to the actual frequencies present in terms of the series. For periodic signals, changes in the relative phase of the partials do not change the perception of sounds that are not too loud.

The purpose of this chapter is twofold. We will first discuss basic aspect of signals and linear systems in the first part. As we will see in subsequent chapters that diffraction as well as optical imaging systems can be modelled as linear systems. In the second part, we introduce the basic properties of Fourier series, Fourier transform as well as the concept of convolution and correlation. Indeed, many modern optical imaging and processing systems can be modelled with the Fourier methods, and Fourier analysis is the main tool to analyze such optical systems. We shall study time signals in one dimension and signals in two dimensions will then be covered. Many of the concepts developed for one-dimensional (1-D) signals and systems apply to two-dimensional (2-D) systems. This chapter also serves to provide important and basic mathematical tools to be used in subsequent chapters.

This chapter is a close companion to the previous one. Here we study the best least squares approximation to periodic functions via trigonometric polynomials. Many of the ideas and results of the previous chapter are repeated in this scenario. They are then expanded to deal with merely square integrable functions. The Fourier transform of periodic functions, and its inverse, is then introduced and studied. Uniform convergence of trigonometric series, under several different smoothness assumptions is then discussed.Trigonometric approximation in periodic Sobolev spaces is then discussed.

A brief introduction to Fourier series and transforms, with explicit expressions for a selected number of functions.

Inner Product Spaces and their Fourier Series are studied in this chapter, including Hilbert Spaces and Adjoint Operators. Convergence of Trigonometric and Square Wave Fourier Series of Integrable Functions are investigated. Theorems connecting Summability of Fourier Series and Summability Kernels are studied. The Radamacher, Walsh, and Haar Systems are defined and studied. The Fourier Transform is introduced.

Analysis of various data sets can be accomplished using techniques based on least-squares methods.For example, linear regression of data determines the best-fit line to the data via a least-squares approach.The same is true for polynomial and regression methods using other basis functions.Curve fitting is used to determine the best-fit line or curve to a particular set of data, while interpolation is used to determine a curve that passes through all of the data points.Polynomial and spline interpolation are discussed.State estimation is covered using techniques based on least-squares methods.

Chapter 3 begins by describing mechanisms of atomic diffusion in crystals, with emphasis on how their rates depend on temperature. Characteristic diffusion lengths and times are explained. The diffusion equation is derived for the chemical composition in space and time, c(r,t). The mathematics for solving the diffusion equation in one dimension are developed by standard approaches with Gaussian functions and error functions. The method of separation of variables is presented for three-dimensional problems in Cartesian and cylindrical coordinates. Typical boundary value problems for diffusion are solved with Fourier series and Bessel functions.

We discuss how to define a basis for a general normed space (a ‘Schauder basis‘). We then consider orthonormal sets in inner-product spaces and orthonormal bases for separable Hilbert spaces. We give a number of conditions that ensure that a particular orthonormal sequence forms an orthonormal basis, and as an example, we discuss the L^2 convergence of Fourier series.

We prove some key results about spaces of continuous functions. First we show that continuous functions on an interval can be uniformly approximated by polynomials (the Weierstrass Approximation Theorem), which has interesting applications to Fourier series. Then we prove the Stone-Weierestrass Theorem, which generalises this to continuous functions on compact metric spaces and other collections of approximating functions. We end with a proof of the Arzelà-Ascoli Theorem.

Using the Baire Category Theorem, we prove the Principle of Uniform Boundedness, which allows us to deduce uniform bounds on collections of bounded linear operators from pointwise properties. We use the powerful corollary known as the “Condensation of Singularities” to show that there are continuous periodic functions whose Fourier series do not converge pointwise everywhere.

Here, we add damping to the harmonic oscillator, and explore the role of the resulting new time scale in the solutions to the equations of motion.Specifically, the ratio of damping to oscillatory time scale can be used to identify very different regimes of motion: under-, critically-, and over-damped.Then driving forces are added, we consider the effect those have on the different flavors of forcing already in place.The main physical example (beyond springs attached to masses in dashpots) is electrical, sinusoidally driven RLC circuits provide a nice, experimentally accessible test case.On the mathematical side, the chapter serves as a thinly-veiled introduction to Fourier series and the Fourier transform.