In Chapter 5, we look at approaches that belong to heuristic algorithms. These methods are derived from observations nature provide. In our argumentation for specific heuristic optimization algorithms, we discuss the local search and the hill climbing problem. One of the outcomes of this discussion is the argument for attempting to avoid cycling during a search. Tabu search optimization is built on this premise where we avoid cycling. An entirely different class of heuristic optimization algorithms are given by Particle Swarm optimization and Ant Colony optimization algorithms. In contrast to Tabu search and local search, the PSO and AC optimization algorithms utilize a number of agents in order to search for optimality. Another multi agent based algorithm is the Genetic algorithm. GA’s are inspired by Darwin’s survival of the fittest principle and use the terminology found in the field of genetics. Additionally, in this chapter we use heuristic optimization to formulate optimum control concepts, including hybrid control using fuzzy logic-based controllers and Matlab scripts to realize each of the heuristic optimization algorithms.

In this chapter, we introduce some of the more popular ML algorithms. Our objective is to provide the basic concepts and main ideas, how to utilize these algorithms using Matlab, and offer some examples. In particular, we discuss essential concepts in feature engineering and how to apply them in Matlab. Support vector machines (SVM), K-nearest neighbor (KNN), linear regression, Naïve Bayes algorithm, and decision trees are introduced and the fundamental underlying mathematics is explained while using Matlab’s corresponding Apps to implement each of these algorithms. A special section on reinforcement learning is included, detailing the key concepts and basic mechanism of this third ML category. In particular, we showcase how to implement reinforcement learning in Matlab as well as make use of some of the Python libraries available online and show how to use reinforcement learning for controller design.

Starting with the perceptron, in Chapter 6 we discuss the functioning, the training, and the use of neural networks. For the different neural network structures, the corresponding script in Matlab is provided and the limitations of the different neural network architectures are discussed. A detailed discussion and the underlying mathematical concept of the Backpropagation learning algorithm is accompanied with simple examples as well as sophisticated implementations using Matlab. Chapter 6 also includesconsiderations on quality measures of trained neural networks, such as the accuracy, recall, specificity, precision, prevalence, and some of the derived quantities such as the F-score and the receiver operating characteristic plot. We also look at the overfitting problem and how to handle it during the neural network training process.

Hybrid systems often try to leverage the advantages of one algorithm with the once of another while minimize its own disadvantages. Having discussed fuzzy logic and neural networks as well as a number of optimization algorithms, Chapter 7 presents several hybrid algorithms that can be used for optimization, controls, and modeling. In particular, we look at neural expert systems, expand these systems to neuro fuzzy systems and adaptive neuro-fuzzy inference systems, which we use for control applications. While revisiting the Mamdani and Sugeno fuzzy inference system, the Tsukamoto fuzzy system as well as different partitioning methods are discussed, such as the grid, the tree and the scatter partitioning. Examples using Matlab FIS app as well as Matlab’s ANFIS editor are used throughout the chapter.

Chapter 3 introduces the concept of the rule base along with the material from Chapter 2 to construct different fuzzy inference systems such as the Mamdani fuzzy inference system or the Sugeno fuzzy inference system. The Takagi-Sugeno fuzzy inference system is used to design fuzzy logic controllers and Lyapunov theory is utilized to investigate the closed-loop system stability of such controllers. Concepts such as local sector nonlinearity, globally asymptotical stability using state-space models are introduced and discussed to fashion controllers for nonlinear systems. Throughout the chapter, Matlab’s FIS editor is used to design fuzzy inference systems and corresponding controllers.

Starting with crisp set theory, fuzzy sets and concepts of fuzzy logic are introduced in Chapter 2. Some of the key operators are discussed and utilized in a number of examples. Membership functions, membership operators, their programming in Matlab, as well as logic operators using membership functions are explained. Along with conditional statements such as fuzzy rules and linguistic variables concepts such as antecedents, consequences and inference are discussed and shown how to implement this type of reasoning in Matlab.

Chapter 1 provides for an introduction to the key concepts of the book, including supervised and unsupervised learning, reinforcement learning, and controls. The objective is to provide an overview of the many methods and algorithms and how they are relate to each other as well as to controls applications.

Based on Chapter 6, in this chapter we expand the discussion of neural networks to include networks that have more than one hidden layer. Common structures such as the convolutional neural network (CNN) or the Long Short-Term Memory network (LSTM) are explained and used along with Matlab’s Deep Network Designer App as well as Matlab script to implement and train such networks. Issues such as the vanishing or exploding gradient, normalization, and training strategies are discussed. Concepts that address overfitting and the vanishing or exploding gradient are introduced, including dropout and regularization. Transfer learning is discussed and showcased using Matlab’s DND App.

In this chapter, we establish the mathematical foundation for hard computing optimization algorithms. We look at the classical optimization approaches and extend our discussion to include iterative methods, which hold a special role in machine learning. In particular, we review the gradient decent method, Newton’s method, the conjugate gradient method and the quasi-Newton’s method. Along with the discussion of these optimization methods, implementation using Matlab script as well as considerations for use in neural network training algorithms are provided. Finally, the Levenberg-Marquardt method is introduced, discussed, and implemented in Matlab script to compare its functioning with the other four iterative algorithms introduced in this chapter.