This chapter starts out by introducing the energy eigenvalue equation – the time-independent Schrödinger equation. Firstly, the notion of energy quantization is introduced by semi-analytical means. A particle is confined within a rectangular well. It is seen, eventually, that the corresponding solutions to the time-independent Schrödinger equation can only exist for a few, specific energies. This phenomenon is also seen for other physical quantities, such as angular momentum and charge. Next, the situation in which the potential experienced by a quantum particle is periodic is given particular attention. It may be studied numerically by minor adjustments of the framework already developed. The case of a periodic potential is important as, to a large extent, it forms the basis for understanding solid state physics. Since the Hamiltonian is Hermitian, its eigenvectors form an orthonormal set in which any state or wave function may be expanded. This is exploited in the last part of the chapter, which is dedicated to determining ground states – the energy eigenstate of minimal energy. This is done in two ways – by using what is called the variational principle and by so-called propagation in imaginary time. These methods are implemented for several examples and compared to full solutions.

In this chapter we cover clustering and regression, looking at two traditional machine learning methods: k-means and linear regression. We briefly discuss how to implement these methods in a non-distributed manner first, to then carefully analyze the bottlenecks of these methods when manipulating big data. This enables us to design global-based solutions based on the DataFrame API of Spark. The key focus is on the principles for designing solutions effectively. Nevertheless, some of the challenges in this chapter are to investigate tools from Spark to speed up the processing even further. k-means is an example of an iterative algorithm, and how to exploit caching in Spark, and we analyze its implementation with both RDD and DataFrame APIs. For linear regression, we first implement the closed form, which involves numerous matrix multiplications and outer products, to simplify the processing in big data. Then, we look at gradient descent. These examples give us the opportunity to expand on the principles of designing a global solution, and also allow us to show how knowing the underlying platform, Spark in this case, well is essential to really maximize the performance.

Chapter 5 starts out with a physics motivation, as well as a mathematical statement of the problems that will be tackled in later sections. Several methods are introduced to solve a single nonlinear equation in one variable: fixed-point iteration, the bisection method, Newton’s method, the secant method, and Ridders’ method. After providing some advice about advantages and disadvantages of each approach, the text then studies how to find zeros of polynomials, employing two different techniques. The sophistication is then increased, by tackling systems of nonlinear equations and examining the corresponding challenges; in addition to Newton’s method, the text derives the equations behind Broyden’s method. A related subject is then broached, minimization in one or several dimensions; this includes the gradient-descent method, as well as detailed analysis of critical points; the second edition includes extensive new material on derivative-free optimization (golden-section search and Powell’s method).The chapter is rounded out by a physics project, the extremization of the action in classical mechanics, and a problem set. The physics project shows Hamilton’s principle in... action, translated into a multidimensional minimization problem.

Many machine learning methods require non-linear optimization, performed by the backward propagation of model errors, with the process complicated by the presence of multiple minima and saddle points. Numerous gradient descent algorithms are available for optimization, including stochastic gradient descent, conjugate gradient, quasi-Newton and non-linear least squares such as Levenberg-Marquardt. In contrast to deterministic optimization, stochastic optimization methods repeatedly introduce randomness during the search process to avoid getting trapped in a local minimum. Evolutionary algorithms, borrowing concepts from evolution to solve optimization problems, include genetic algorithm and differential evolution.

We prove the equivalence of the solution to a linear system of equations with an HPD matrix to the problem of quadratic minimization. With the help of this equivalence, we study the minimization of a quadratic energy and introduce gradient descent methods with exact and approximate line search, we study the preconditioned steepest descent method. We introduce the conjugate gradient method, with preconditioning, as a Galerkin approximation over Krylov subspaces and show its convergence. For systems with non HPD matrices we discuss the GMRES method.

In this section, we discuss fundamental methods, mostly based on gradient information, that yield descent, that is, the function value decreases at each iteration. We start with the most basic method, the steepest-descent method, analyzing its convergence under different convexity/nonconvexity assumptions on the objective function. We then discuss more general descent methods, based on descent directions other than the negative gradient, showing conditions on the search direction and the steplength that allow convergence results to be proved. We also discuss a method that also makes use of Hessian information, showing that it can find a point satisfying approximate second-order optimality conditions and finding an upper bound on the number of iterations required to do so. We then discuss mirror descent, a class of gradient methods based on more general distance metrics that are particularly useful in optimizing over the unit simplex – a problem that arises often in data science. We conclude by discussing the PL condition, a generalization of the strong convexity condition that allows linear convergence rates to be proved.

This chapter is not intended to be a complete discussion of machine learning. We concentrate on a small number of ideas, and emphasize how to deal with very large data sets. Especially important is how we exploit parallelism to build models of the data. We consider the classical “perceptron” approach to learning a data classifier, where a hyperplane that separates two classes is sought. Then, we look at more modern techniques involving support-vector machines. Similar to perceptrons, these methods look for hyperplanes that best divide the classes, so that few, if any, members of the training set lie close to the hyperplane. We next consider nearest-neighbor techniques, where data is classified according to the class(es) of their nearest neighbors in some space. We end with a discussion of decision trees, which are branching programs for predicting the class of an example.