Consensus-based optimisation (CBO) is a versatile multi-particle metaheuristic optimisation method suitable for performing non-convex and non-smooth global optimisations in high dimensions. It has proven effective in various applications while at the same time being amenable to a theoretical convergence analysis. In this paper, we explore a variant of CBO, which incorporates truncated noise in order to enhance the well-behavedness of the statistics of the law of the dynamics. By introducing this additional truncation in the noise term of the CBO dynamics, we achieve that, in contrast to the original version, higher moments of the law of the particle system can be effectively bounded. As a result, our proposed variant exhibits enhanced convergence performance, allowing in particular for wider flexibility in choosing the noise parameter of the method as we confirm experimentally. By analysing the time evolution of the Wasserstein-$2$ distance between the empirical measure of the interacting particle system and the global minimiser of the objective function, we rigorously prove convergence in expectation of the proposed CBO variant requiring only minimal assumptions on the objective function and on the initialisation. Numerical evidences demonstrate the benefit of truncating the noise in CBO.

In this paper, we study consensus-based optimisation (CBO), a versatile, flexible and customisable optimisation method suitable for performing nonconvex and nonsmooth global optimisations in high dimensions. CBO is a multi-particle metaheuristic, which is effective in various applications and at the same time amenable to theoretical analysis thanks to its minimalistic design. The underlying dynamics, however, is flexible enough to incorporate different mechanisms widely used in evolutionary computation and machine learning, as we show by analysing a variant of CBO which makes use of memory effects and gradient information. We rigorously prove that this dynamics converges to a global minimiser of the objective function in mean-field law for a vast class of functions under minimal assumptions on the initialisation of the method. The proof in particular reveals how to leverage further, in some applications advantageous, forces in the dynamics without loosing provable global convergence. To demonstrate the benefit of the herein investigated memory effects and gradient information in certain applications, we present numerical evidence for the superiority of this CBO variant in applications such as machine learning and compressed sensing, which en passant widen the scope of applications of CBO.

Loss functions with a large number of saddle points are one of the major obstacles for training modern machine learning (ML) models efficiently. First-order methods such as gradient descent (GD) are usually the methods of choice for training ML models. However, these methods converge to saddle points for certain choices of initial guesses. In this paper, we propose a modification of the recently proposed Laplacian smoothing gradient descent (LSGD) [Osher et al., arXiv:1806.06317], called modified LSGD (mLSGD), and demonstrate its potential to avoid saddle points without sacrificing the convergence rate. Our analysis is based on the attraction region, formed by all starting points for which the considered numerical scheme converges to a saddle point. We investigate the attraction region’s dimension both analytically and numerically. For a canonical class of quadratic functions, we show that the dimension of the attraction region for mLSGD is $\lfloor (n-1)/2\rfloor$, and hence it is significantly smaller than that of GD whose dimension is $n-1$.

As a continuation of previous work of the first author with Ranjbar [‘A variational inequality in complete CAT(0) spaces’, J. Fixed Point Theory Appl. 17 (2015), 557–574] on a special form of variational inequalities in Hadamard spaces, in this paper we study equilibrium problems in Hadamard spaces, which extend variational inequalities and many other problems in nonlinear analysis. In this paper, first we study the existence of solutions of equilibrium problems associated with pseudo-monotone bifunctions with suitable conditions on the bifunctions in Hadamard spaces. Then, to approximate an equilibrium point, we consider the proximal point algorithm for pseudo-monotone bifunctions. We prove existence of the sequence generated by the algorithm in several cases in Hadamard spaces. Next, we introduce the resolvent of a bifunction in Hadamard spaces. We prove convergence of the resolvent to an equilibrium point. We also prove $\triangle$-convergence of the sequence generated by the proximal point algorithm to an equilibrium point of the pseudo-monotone bifunction and also the strong convergence under additional assumptions on the bifunction. Finally, we study a regularization of Halpern type and prove the strong convergence of the generated sequence to an equilibrium point without any additional assumption on the pseudo-monotone bifunction. Some examples in fixed point theory and convex minimization are also presented.

This paper presents topology optimization of capillary, the typical two-phase flow with immiscible fluids, where the level set method and diffuse-interface model are combined to implement the proposed method. The two-phase flow is described by the diffuse-interface model with essential no slip condition imposed on the wall, where the singularity at the contact line is regularized by the molecular diffusion at the interface between two immiscible fluids. The level set method is utilized to express the fluid and solid phases in the flows and the wall energy at the implicit fluid-solid interface. Based on the variational procedure for the total free energy of two-phase flow, the Cahn-Hilliard equations for the diffuse-interface model are modified for the two-phase flow with implicit boundary expressed by the level set method. Then the topology optimization problem for the two-phase flow is constructed for the cost functional with general formulation. The sensitivity analysis is implemented by using the continuous adjoint method. The level set function is evolved by solving the Hamilton-Jacobian equation, and numerical test is carried out for capillary to demonstrate the robustness of the proposed topology optimization method. It is straightforward to extend this proposed method into the other two-phase flows with two immiscible fluids.

The pricing model for American lookback options can be characterised as a two-dimensional free boundary problem. The main challenge in this problem is the free boundary, which is also the main concern for financial investors. We use a standard technique to reduce the pricing model to a one-dimensional linear complementarity problem on a bounded domain and obtain a corresponding variational inequality. The inequality is discretised by finite differences and finite elements in the temporal and spatial directions, respectively. By enforcing inequality constraints related to the options using Lagrange multipliers, the discretised variational inequality is reformulated as a set of semi-smooth equations, which are solved by a primal-dual active set method. One of the major advantages of our algorithm is that we can obtain the option values and the free boundary simultaneously, and numerical simulations show that our approach is as efficient as some other methods.

The hybrid variational model for restoration of texture images corrupted by blur and Gaussian noise we consider combines total variation regularisation and a fractional-order regularisation, and is solved by an alternating minimisation direction algorithm. Numerical experiments demonstrate the advantage of this model over the adaptive fractional-order variational model in image quality and computational time.

We consider the total curvature of graphs of curves in high-codimension Euclidean space. We introduce the corresponding relaxed energy functional and prove an explicit representation formula. In the case of continuous Cartesian curves, i.e. of graphs cu of continuous functions u on an interval, we show that the relaxed energy is finite if and only if the curve cu has bounded variation and finite total curvature. In this case, moreover, the total curvature does not depend on the Cantor part of the derivative of u. We treat the wider class of graphs of one-dimensional functions of bounded variation, and we prove that the relaxed energy is given by the sum of the length and total curvature of the new curve obtained by closing the holes in cu generated by jumps of u with vertical segments.

We propose a new two-phase method for reconstruction of blurred images corrupted by impulse noise. In the first phase, we use a noise detector to identify the pixels that are contaminated by noise, and then, in the second phase, we reconstruct the noisy pixels by solving an equality constrained total variation minimization problem that preserves the exact values of the noise-free pixels. For images that are only corrupted by impulse noise (i.e., not blurred) we apply the semismooth Newton's method to a reduced problem, and if the images are also blurred, we solve the equality constrained reconstruction problem using a first-order primal-dual algorithm. The proposed model improves the computational efficiency (in the denoising case) and has the advantage of being regularization parameter-free. Our numerical results suggest that the method is competitive in terms of its restoration capabilities with respect to the other two-phase methods.

Retinex theory explains how the human visual system perceives colors. The goal of retinex is to decompose the reflectance and the illumination from the given images and thereby compensating for non-uniform lighting. The existing methods for retinex usually use a single image with a fixed exposure to restore the reflectance of the image. In this paper, we propose a variational model for retinex problem by utilizing multi-exposure images of a given scene. The existence and uniqueness of the solutions of the proposed model have been elaborated. An alternating minimization method is constructed to solve the proposed model and its convergence is also demonstrated. The experimental results show that the proposed method is effective for reflectance recovery in retinex problem.

This paper introduces a two-stage model for multi-channel image segmentation, which is motivated by minimal surface theory. Indeed, in the first stage, we acquire a smooth solution u from a convex variational model related to minimal surface property and different data fidelity terms are considered. This minimization problem is solved efficiently by the classical primal-dual approach. In the second stage, we adopt thresholding to segment the smoothed image u. Here, instead of using K-means to determine the thresholds, we propose a more stable hill-climbing procedure to locate the peaks on the 3D histogram of u as thresholds, in the meantime, this algorithm can also detect the number of segments. Finally, numerical results demonstrate that the proposed method is very robust against noise and superior to other image segmentation approaches.

In this work, we develop an adaptive algorithm for solving elliptic optimal control problems with simultaneously appearing state and control constraints. The algorithm combines a Moreau-Yosida technique for handling state constraints with a semi-smooth Newton method for solving the optimality systems of the regularized sub-problems. The state and adjoint variables are discretized using continuous piecewise linear finite elements while a variational discretization concept is applied for the control. To perform the adaptive mesh refinements cycle we derive local error estimators which extend the goal-oriented error approach to our setting. The performance of the overall adaptive solver is assessed by numerical examples.

In this work, we develop a minimum action method (MAM) with optimal linear time scaling, called tMAM for short. The main idea is to relax the integration time as a functional of the transition path through optimal linear time scaling such that a direct optimization of the integration time is not required. The Feidlin-Wentzell action functional is discretized by finite elements, based on which h-type adaptivity is introduced to tMAM. The adaptive tMAM does not require reparametrization for the transition path. It can be applied to deal with quasi-potential: 1) When the minimal action path is subject to an infinite integration time due to critical points, tMAM with a uniform mesh converges algebraically at a lower rate than the optimal one. However, the adaptive tMAM can recover the optimal convergence rate. 2) When the minimal action path is subject to a finite integration time, tMAM with a uniform mesh converges at the optimal rate since the problem is not singular, and the optimal integration time can be obtained directly from the minimal action path. Numerical experiments have been implemented for both SODE and SPDE examples.

An inverse problem of identifying inhomogeneity or crack in the workpiece made of nonlinear magnetic material is investigated. To recover the shape from the local measurements, a piecewise constant level set algorithm is proposed. By means of the Lagrangian multiplier method, we derive the first variation w.r.t the piecewise constant level set function and obtain the descent direction by the adjoint variable method. Numerical results show the robustness and effectiveness of our algorithm applied to reconstruct some complex shapes.

An efficient numerical method is proposed for the valuation of American options via the Black-Scholes variational inequality. A far field boundary condition is employed to truncate the unbounded domain problem to produce the bounded domain problem with the associated variational inequality, to which our finite element method is applied. We prove that the matrix involved in the finite element method is symmetric and positive definite, and solve the discretized variational inequality by the projection and contraction method. Numerical experiments are conducted that demonstrate the superior performance of our method, in comparison with earlier methods.

We propose a new nonrigid registration algorithm which is based on the optimal control approach. In our previously proposed methods, the Jacobian determinant and the curl vector were used as control functions. In this algorithm, we use a new set of control functions. A main advantage of using the new controls is that the positivity and normalization of the Jacobian determinant are satisfied automatically. Numerical results on large deformation brain images are provided to show the accuracy and efficiency of the algorithm.

In this paper, we propose a fast proximity point algorithm and apply it to total variation (TV) based image restoration. The novel method is derived from the idea of establishing a general proximity point operator framework based on which new first-order schemes for total variation (TV) based image restoration have been proposed. Many current algorithms for TV-based image restoration, such as Chambolle’s projection algorithm, the split Bregman algorithm, the Bermúdez-Moreno algorithm, the Jia-Zhao denoising algorithm, and the fixed point algorithm, can be viewed as special cases of the new first-order schemes. Moreover, the convergence of the new algorithm has been analyzed at length. Finally, we make comparisons with the split Bregman algorithm which is one of the best algorithms for solving TV-based image restoration at present. Numerical experiments illustrate the efficiency of the proposed algorithms.

In this paper we present recent developments concerning a Cell-Centered Arbitrary Lagrangian Eulerian (CCALE) strategy using the Moment Of Fluid (MOF) interface reconstruction for the numerical simulation of multi-material compressible fluid flows on unstructured grids in cylindrical geometries. Especially, our attention is focused here on the following points. First, we propose a new formulation of the scheme used during the Lagrangian phase in the particular case of axisymmetric geometries. Then, the MOF method is considered for multi-interface reconstruction in cylindrical geometry. Subsequently, a method devoted to the rezoning of polar meshes is detailed. Finally, a generalization of the hybrid remapping to cylindrical geometries is presented. These explorations are validated by mean of several test cases using unstructured grid that clearly illustrate the robustness and accuracy of the new method.