In this chapter, we establish the mathematical foundation for hard computing optimization algorithms. We look at the classical optimization approaches and extend our discussion to include iterative methods, which hold a special role in machine learning. In particular, we review the gradient decent method, Newton’s method, the conjugate gradient method and the quasi-Newton’s method. Along with the discussion of these optimization methods, implementation using Matlab script as well as considerations for use in neural network training algorithms are provided. Finally, the Levenberg-Marquardt method is introduced, discussed, and implemented in Matlab script to compare its functioning with the other four iterative algorithms introduced in this chapter.

Now that we are acquainted with both mechanical and electrical systems, we would like to connect these two physical domains so that we are able to model, analyze, and design electromechanical systems. Such systems include many important sensors, actuators, and devices including motors, generators, loudspeakers, accelerometers, industrial machinery, vehicle subsystems, and more. Thus, following our earlier discussions, we will begin by reviewing the fundamental physical effects that are leveraged in these devices, and work toward extending our mechanical modeling approach to this class of multi-physics problems.

Following our discussion of translational and rotational mechanical systems, we will now extend our mechanical modeling approach (via the system decomposition technique) to electrical circuits. As in Ch. 2, we begin by considering the fundamental elements from which many electrical systems are built, and we review the corresponding mathematical element laws and electrical circuit diagrams. Then, moving forward, the focus shifts toward the rules that govern the interconnection of these elements as complete circuits and subsystems of series and parallel elements. Through these discussions, we will find that, just as Newton’s second law guided our modeling approach for mechanical systems, our treatment of electrical systems will follow Kirchhoff’s voltage and current laws.

To motivate our interest in the broad field of , let us begin by first dissociating the descriptor “dynamic” and focus solely on the objects of our interest, the “systems.” Throughout this text, we will considerof many kinds. To understand the true scope of potential applications for the tools that we will develop, it is advantageous to establish a clear vision of what qualifies as a system and why. To ensure suitability across many domains, we will prefer the flexibility of a loose conceptual definition, by which the term system will refer to any collection of elements (i.e., physical or mathematical) that have cause-and-effect relationships. Then, reintroducing “dynamic” as a reference to changes that occur over time, the field of dynamic systems encompasses the study of cause-and-effect relationships that propagate changes over time.

In this chapter, we continue our discussion of feedback control systems from the perspective of frequency-domain system properties, which were briefly introduced in Ch. 6. Accordingly, we begin this chapter with a comprehensive review of frequency-domain analysis techniques relating to the response of systems to inputs, or excitations, of the form.

Recall from our discussions in Part II that dynamic systems can be represented through various mathematical modeling strategies, including systems of ordinary differential equations, systems of Laplace-domain equations, transfer functions, functional block diagrams, and physical models. Through our review of the Laplace domain, we found that the zero-state input–output behavior of dynamic systems could be represented, conveniently, as the ratio of the system’s Laplace-domain output and its Laplace-domain input. These ratios, or transfer functions, simplified our analysis of interconnected systems and enabled the design of feedback control systems in Part III. However, while the Laplace-domain and transfer functions are undoubtedly powerful and useful tools for systems with a single input and a single output, known as(SISO) systems, the implementation of transfer functions can become cumbersome and complex for systems with multiple inputs and multiple outputs, known as(MIMO) systems.

In this chapter we will discuss an important graphical tool in feedback system design known as the . As the name suggests, this technique allows us to graph the closed-loop pole locations (i.e., the roots of the characteristic equation) on the complex plane as a function of a selected parameter (e.g., as the value of a resistor is varied or as controller gains are changed). Thus, not only does this technique allow us to extend our concept of stability analysis from the more mathematical Routh–Hurwitz approach in Ch. 8 but it also allows us to visualize the stability of a system and select its parameters to achieve desired system characteristics.

Up to this point, we have developed the necessary tools to generate mathematical models of dynamic systems. The discussions in this area have covered physical systems comprising mechanical, electrical, thermal, and fluidic components, and, further, we have shown how some fundamental multi-domain problems may be addressed by the mechanical modeling approach. Under this paradigm, all the tools and techniques we have discussed inevitably lead to mathematical representations in the form of coupled ODEs of motion. Through these discussions, we have also demonstrated that the mechanical modeling approach does not necessarily produce a linear governing equation and may often result in mathematical models that have nonlinear terms. However, since the solution techniques for linear and nonlinear systems differ, all the tools we will discuss in Chs. 6–10 assume that the equations of motion are linear.

In Ch. 5, we introduced the concept of state-space models as an alternative to traditional systems of nth-order governing equations. Unlike governing equations derived from first principles, such as those discussed in Chs. 2–4, state-space models always comprise first-order ODEs and can be analyzed and solved using linear algebra rather than higher-order ODEs.