Currently, most wheel-legged robots need to complete the switching of the wheel-and-leg modal in a stationary state, and the existing algorithms of statically switching the wheel-leg modal cannot meet the control requirements of multimodal switching dynamically for robots. In this paper, to achieve efficient switching of the wheel-and-leg modal for a quadruped robot, the novel transformable mechanism is designed. Then, a multimodal coordination operation control framework based on multiple algorithms is presented, incorporating the minimum foot force distribution method (algorithm No.1), the minimum joint torque distribution method (algorithm No.2), and the method of combining the single rigid body dynamic model with quadratic programming (algorithm No.3). In the process of switching wheel-leg modal dynamically, the existing algorithm No.3 is prone to produce the wrong optimal force due to the change of the whole-body rotational inertia. Therefore, an improved algorithm No.1 and algorithm No.2 are proposed, which do not consider the change in the body’s inertia. The control effects of the three algorithms are compared and analyzed by simulation. The results show that algorithm No.3 can maintain a small error in attitude angle and speed tracking regardless of whether the robot is under multilegged support or omnidirectional walking compared to the other two algorithms. However, proposed algorithms No.1 and No.2 can more accurately track the target speed when the robot is walking with wheels raising and falling. Finally, a multi-algorithm combination control scheme formulated based on the above control effects has been demonstrated to be effective for the dynamic switching of the wheel-and-leg modal.

We design a scheme for laser-inertial odometry and mapping with bundle adjustment (BA-LIOM), which can greatly mitigate the problem of undesired ground warping due to sparsity of laser scans and significantly reduce odometry drift. Specifically, an Inertial measurement unit (IMU)-assisted adaptive voxel map initialization algorithm is proposed and elaborately integrated with the existing framework LIO-SAM, allowing for accurate registration in the beginning of the localization and mapping process. In addition, to accommodate to fast-moving and structure-less scenarios, we design a tightly coupled odometry, which jointly optimizes both the IMU preintegration constraints and scan matching with adaptive voxel maps. The voxels (edge and plane, respectively) are updated with BA optimization. And then the accurate mapping result is obtained by performing local BA. The proposed BA-LIOM is thoroughly assessed using datasets collected from multiple platforms over a variety of environments. Experimental results show the superiority of BA-LIOM over the state-of-the-art methods in robustness and precision, especially for large-scale scenarios. BA-LIOM improves the accuracy of localization by $61\%$ and $73\%$ on the buildings and lawn datasets, respectively, and has a $29\%$ accuracy improvement over LIO-SAM on the KITTI datasets. A supplementary video can be accessed at https://youtu.be/5l4ZFhTc2sw.

Coverage path planning (CPP) is a subfield of path planning problems in which free areas of a given domain must be visited by a robot at least once while avoiding obstacles. In some situations, the path may be optimized for one or more criteria such as total distance traveled, number of turns, and total area covered by the robot. Accordingly, the CPP problem has been formulated as a multi-objective optimization (MOO) problem, which turns out to be a challenging discrete optimization problem, hence conventional MOO algorithms like Non-dominated Sorting Genetic Algorithm-2 (NSGA-II) do not work as it is. This study implements a modified NSGA-II to solve the MOO problem of CPP for a mobile robot. In this paper, the proposed method adopted two objective functions: (1) the total distance traveled by the robot and (2) the number of turns taken by the robot. The two objective functions are used to calculate energy consumption. The proposed method is compared to the hybrid genetic algorithm (HGA) and the traditional genetic algorithm (TGA) in a rectilinear environment containing obstacles of various complex shapes. In addition, the results of the proposed algorithm are compared to those generated by HGA, TGA, oriented rectilinear decomposition, and spatial cell diffusion and family of spanning tree coverage in existing research papers. The results of all comparisons indicate that the proposed algorithm outperformed the existing algorithms by reducing energy consumption by 5 to 60%. This paper provides the facility to operate the robot in different modes.

In this paper, we consider the problem of contact parameters (slippage and sinkage) estimation for multi-modal robot locomotion on granular terrains. To describe the contact events in the same framework for robots operated at different modes (e.g., wheel, leg), we propose a unified description of contact parameters for multi-modal robots. We also provide a parameter estimation method for multi-modal robots based on CNN and DWT (discrete wavelet transformation) techniques and verify its effectiveness over different types of granular terrains. Besides motion modes, this paper also considers the influence of slope angles and the robot’s handing angles over contact parameters. Through comparison and analysis of the prediction results, our method can not only effectively predict the contact parameters of multi-modal robot locomotion on a granular medium (better than $96\%$ accuracy) but also achieves the same or better performance when compared to other (direct) contact measurement methods designed for individual motion modes, that is, single-modal robots such as quadruped robots and mars rovers. Our proposed unified contact parameter estimation method can be useful for studying the interaction mechanics between multi-modal robots and granular terrains as well as terrain classification tasks due to its superior sensitivity which is analyzed in the experiments.

At present, the research of soft crawling robot pays more attention to the material manufacturing, but neglects the robot modeling. The high degree of freedom of the soft crawling robot makes it more difficult to establish its motion model and analyzes its motion performance. The centimeter-level wireless driven soft crawling robot has great advantages and application scenarios in narrow space exploration. Therefore, this paper proposed a soft robot driven by magnetism to crawling inchworm. By studying the crawling behavior of inchworm and the characteristics of flexible materials driven by magnetism, the structure of the soft robot was designed and the motion model of inchworm was established. The motion model is analyzed and simulated, the structure size of the robot is optimized, and the effectiveness of the model is verified by experiments. The robot’s crawling motion is realized by coupling the structure of the robot’s torso and legs with the flexible magnetic film. Driven by the alternating magnetic field, the maximum motion speed of the robot is 28.24 mm/s. At the same time, the robot can also move in narrow space such as pipes, which can satisfy the centimeter-level space detection and ensure the high efficiency, providing a new idea for narrow space detection.

With the rise of deep reinforcement learning (RL) methods, many complex robotic manipulation tasks are being solved. However, harnessing the full power of deep learning requires large datasets. Online RL does not suit itself readily into this paradigm due to costly and time-consuming agent-environment interaction. Therefore, many offline RL algorithms have recently been proposed to learn robotic tasks. But mainly, all such methods focus on a single-task or multitask learning, which requires retraining whenever we need to learn a new task. Continuously learning tasks without forgetting previous knowledge combined with the power of offline deep RL would allow us to scale the number of tasks by adding them one after another. This paper investigates the effectiveness of regularisation-based methods like synaptic intelligence for sequentially learning image-based robotic manipulation tasks in an offline-RL setup. We evaluate the performance of this combined framework against common challenges of sequential learning: catastrophic forgetting and forward knowledge transfer. We performed experiments with different task combinations to analyse the effect of task ordering. We also investigated the effect of the number of object configurations and the density of robot trajectories. We found that learning tasks sequentially helps in the retention of knowledge from previous tasks, thereby reducing the time required to learn a new task. Regularisation-based approaches for continuous learning, like the synaptic intelligence method, help mitigate catastrophic forgetting but have shown only limited transfer of knowledge from previous tasks.

Human gait trajectory prediction is a long-standing research topic in human–machine interaction. However, there are two shortcomings in the current gait trajectory prediction technology. The first shortcoming is that the neural network model of gait prediction only predicts dozens of future time frames of gait trajectory. The second shortcoming is that the gait prediction neural network model is uninterpretable. We propose the Interpretable-Concatenation former (IC-former) model, which can predict long-term gait trajectories and explain the prediction results by quantifying the importance of data at different positions in the input sequence. Experiments prove that the IC-former model we proposed not only makes a breakthrough in prediction accuracy but also successfully explains the data basis of the prediction.

This paper explores and experimentally compares the effectiveness of robot-stopping approaches based on the speed and separation monitoring for improving fluency in collaborative robotics. In the compared approaches, a supervisory controller checks the distance between the bounding volumes enclosing human operator and robot and prevents potential collisions by determining the robot’s stop time and triggering a stop trajectory if necessary. The methods are tested on a Franka Emika robot with 7 degrees of freedom, involving 27 volunteer participants, who are asked to walk along assigned paths to cyclically intrude the robot workspace, while the manipulator is working. The experimental results show that scaling online the dynamic safety zones is beneficial for improving fluency of human-robot collaboration, showing significant statistical differences with respect to alternative approaches.

An online pattern recognition method of lower limb movements is proposed based on the personalized surface electromyography (sEMG) signals, and the corresponding experimental researches are performed in the rehabilitation training. Further, a wireless wearable acquisition instrument is used. Based on this instrument, a host computer for the personal online recognition and real-time control of rehabilitation training is developed. Three time-domain features and two features in the nonlinear dynamics are selected as the joint set of the characteristic values for the sEMG signals. Then a particle swarm optimization (PSO) algorithm is used to optimize the feature channels, and a k-nearest neighbor (KNN) algorithm and the extreme learning machine (ELM) algorithm are combined to classify and recognize individual sample data. Based on the multi-pose lower limb rehabilitation robot, the real-time motion recognition and the corresponding rehabilitation training are carried out by using the online personalized classifier. The experimental results of eight subjects indicate that it takes only 6 min to build an online personalized classifier for the four types of the lower limb movements. The recognition between switches of different rehabilitation training movements is timely and accurate, with an average recognition accuracy of more than 95%. These results demonstrate that this system has a strong practicability.

The intricate water-land intermingled nature of wild environments necessitates robots to exhibit multimodal cross-domain mobility capabilities. This paper introduces a novel wheel-spoke-paddle hybrid amphibious robot (WSP-bot) that can operate on flat and rough terrains, water surfaces, and water-land transitional zones. The proposed robot relies on a propulsion mechanism called transformable wheel-spoke-paddle (WSP), which combines the stability of wheeled robots with the obstacle-climbing capability of legged robots, while also providing additional aquatic mobility. The utilization of a crank-slider-based transformation mechanism enables seamless switching between multiple motion modes. An analysis of mode transition and ground motion in spoke mode was conducted, along with an investigation of its obstacle-crossing capability. Simulations were performed for mode transition, ground locomotion, and obstacle-crossing, as well as propulsion of a single WSP module on water. Based on the above work, a prototype robot was manufactured. Prototype tests, including mode transition and mobility tests on land and water surfaces under multimodal states, confirmed the effectiveness of the proposed WSP-bot.

This paper presents a compliant variable admittance adaptive fixed-time sliding mode control (SMC) algorithm for trajectory tracking of robotic manipulators. Specifically, a compliant variable admittance algorithm and an adaptive fixed-time SMC algorithm are combined to construct a double-loop control structure. In the outer loop, the variable admittance algorithm is developed to adjust admittance parameters during a collision to minimize the collision time, which gives the robot compliance property and reduce the rigid collision influence. Then, by employing the Lyapunov theory and the fixed-time stability theory, a new nonsingular sliding mode manifold is proposed and an adaptive fixed-time SMC algorithm is presented in the inner loop. More precisely, this approach enables rapid convergence, enhanced steady-state tracking precision, and a settling time that is independent of system initial states. As a result, the effectiveness and improved performance of the proposed algorithm are demonstrated through extensive simulations and experimental results.

Heavy-duty hexapod robots are well-suited for physical transportation, disaster relief, and resource exploration. The immense locomotion capabilities conferred by the six appendages of these systems enable traversal over unstructured and challenging terrain. However, tipping can be a serious concern when moving with a tripod gait in these challenging environments, which may cause irreversible consequences such as compromised movement control and potential damage. In this paper, we focus on heavy-duty hexapod robot sideline tipping judgment and recovery during tripod gait motion, and a novel sideline tipping judgment and recovery method is proposed by adjusting an optimal swinging leg to the stance state. Considering the locomotion environments, motion mode, and tipping analysis, the robot’s stability margin is quantified, and the tipping event is evaluated by the Force Angle Stability Measure (FASM). The recovery method is initiated upon detecting that the robot is tipping, which involves the selection of an adjustment leg and the determination of an optimal foothold. Since the FASM is based on the foot force and robot center of gravity (CoG), the stability margin quantification expression is reformulated to the constraint form of quadratic programming (QP). Furthermore, a foot force distribution method, integrating stability margin considerations into the QP model, has been devised to ensure post-adjustment stability of the landing leg. Experiments on tipping judgment and recovery demonstrate the effectiveness of the proposed approaches on tipping judgment and recovery.

The integration of camera and LiDAR technologies has the potential to significantly enhance construction robots’ perception capabilities by providing complementary construction information. Structured light cameras (SLCs) are a desirable alternative as they provide comprehensive information on construction defects. However, fusing these two types of information depends largely on the sensors’ relative positions, which can only be established through extrinsic calibration. This paper introduces a novel calibration algorithm considering a customized board for SLCs and repetitive LiDARs, which are designed to facilitate the automation of construction robots. The calibration board is equipped with four symmetrically distributed hemispheres, whose centers are obtained by fitting the spheres and adoption with the geometric constraints. Subsequently, the spherical centers serve as reference features to estimate the relationship between the sensors. These distinctive features enable our proposed method to only require one calibration board pose and minimize human intervention. We conducted both simulation and real-world experiments to assess the performance of our algorithm. And the results demonstrate that our method exhibits enhanced accuracy and robustness.

This article proposes a control method for underactuated cartpole systems using semi-implicit cascaded proportional-derivative (PD) controller. The proposed controller is composed of two conventional PD controllers, which stabilizes the pole and the cart second-order dynamics respectively. The first PD controller is realized by transforming the pole dynamics into a virtual PD controller, with the coupling term exploited as the internal tracking target for the cart dynamics. Then, the second PD controller manipulates the cart dynamics to track that internal target. The solution to the internal tracking target relies on an equation set and features a semi-implicit process, which exploits the internal dynamics of the system. Besides, the design of second PD controller relies on the parameters of the first PD controller in a cascaded manner. A stability analysis approach based on Jacobian matrix is proposed and implemented for this fourth-order system. The proposed method is simple in design and intuitive to comprehend. The simulation results illustrate the superiority of proposed method compared with conventional double-loop PD controller in terms of convergence, with the theoretical conclusion of at least locally asymptotic stability.

Currently, workers in sand casting face harsh environments and the operation safety is poor. Existing pouring robots have insufficient stability and load-bearing capacity and cannot perform intelligent pouring according to the demand of pouring process. In this paper, a hybrid pouring robot is proposed to solve these limitations, and a vision-based hardware-in-the-loop (HIL) control technology is designed to achieve the real-time control problems of simulated pouring and pouring process. Firstly, based on the pouring mechanism and the motion demand of ladle, a hybrid pouring robot with a 2UPR-2RPU parallel mechanism as the main body is designed. And the equivalent hybrid kinematic model was established by using Eulerian method and differential motion. Subsequently, a motion control strategy based on HIL simulation technique was designed and presented. The working space of the robot was obtained through simulation experiments to meet the usage requirements. And the stability of the robot was tested through the key motion parameters of the robot joints. Based on the analysis of pouring quality and trajectory, optimal dynamic parameters for the experimental prototype are obtained through water simulation experiments, the pouring liquid height area is 35–40 cm, the average flow rate of pouring liquid is 112 cm3/s, and the ladle tilting speed is 0.0182 rad/s. Experimental results validate the reasonableness of the designed pouring robot structure. Its control system realizes the coordinated movement of each branch chain to complete the pouring tasks with different variable parameters. Consequently, the designed pouring robot will significantly enhance the automation level of the casting industry.

Unmanned aerial vehicles (UAVs) possess fast-moving abilities and have been used in various tasks in the past decades. However, their performances are still restricted by insufficient endurance and confined environments. Intuitively, combining other locomotion modes with UAVs, such as diving and driving, would be an appropriate idea to improve the robot’s adaptability and solve the endurance problem. Recently, the terrestrial/aerial hybrid robots have drawn the researchers’ eyes for their outstanding performances, which can deploy flight mode to traverse insurmountable terrains and ground mode to increase endurance and realize detailed searches. Therefore, this paper developed the autonomous quadrotor tilting hybrid robot (AQT-HR) to achieve terrestrial/aerial dual-modal mobility and verified that the robot delivers high energy efficiency. The AQT-HR can achieve flying and driving through a quadrotor tilting mechanism, which can alter one single driving force into different directions. Furthermore, the dynamic models of the hybrid robot’s aerial and ground locomotion are derived and introduced into the model-feedforward PID control algorithm for improving the robot’s flying stability. Finally, we conducted some mobility tests and experiments about traversing obstacles to demonstrate that the proposed hybrid robot can realize autonomous mode switching and perform a low energy consumption in ground movement mode.

Robots with multi-sensors always have a problem of weak pairing among different modals of the collected information produced by multi-sensors, which leads to a bad perception performance during robot interaction. To solve this problem, this paper proposes a Force Vision Sight (FVSight) sensor, which utilizes a distributed flexible tactile sensing array integrated with a vision unit. This innovative approach aims to enhance the overall perceptual capabilities for object recognition. The core idea is using one perceptual layer to trigger both tactile images and force-tactile arrays. It allows the two heterogeneous tactile modal information to be consistent in the temporal and spatial dimensions, thus solving the problem of weak pairing between visual and tactile data. Two experiments are specially designed, namely object classification and slip detection. A dataset containing 27 objects with deep presses and shallow presses is collected for classification, and then 20 slip experiments on three objects are conducted. The determination of slip and stationary state is accurately obtained by covariance operation on the tactile data. The experimental results show the reliability of generated multimodal data and the effectiveness of our proposed FVSight sensor.

Most of the currently available three-degree-of-freedom manipulators are light load and cannot achieve full continuous rotation; given this, we designed a heavy-load manipulator that achieves unrestricted and continuous rotation. Due to manufacturing and assembly errors, parameter deviations between the real manipulator and its underlying theoretical model were unavoidable. Because of the lack of high-precision, high-frequency, and real-time closed-loop detection methods, we proposed a type of kinematics calibration of parameterized ant colony optimization and feedforward control methods. This was done to achieve high-precision motion control. First, an error model combining structural parameters and joint output angles was established, and the global sensitivity of each error source was analyzed to distinguish both primary and secondary sources. Based on the measured data of a laser tracker, the ant colony optimization was then used to identify six error sources. This resulted in both link length and joint driving errors of the designed manipulator. As it is a type of systematic error, the rounding error of the theoretical trajectory was carefully analyzed, and feedforward control methods with different coefficients were designed to further improve positioning accuracy based on the kinematic calibration. Experimental results showed that the proposed kinematic calibration and feedforward control methods achieved relatively precise motion control for the designed manipulator.

The language-guided visual robotic grasping task focuses on enabling robots to grasp objects based on human language instructions. However, real-world human-robot collaboration tasks often involve situations with ambiguous language instructions and complex scenarios. These challenges arise in the understanding of linguistic queries, discrimination of key concepts in visual and language information, and generation of executable grasping configurations for the robot’s end-effector. To overcome these challenges, we propose a novel multi-modal transformer-based framework in this study, which assists robots in localizing spatial interactions of objects using text queries and visual sensing. This framework facilitates object grasping in accordance with human instructions. Our developed framework consists of two main components. First, a visual-linguistic transformer encoder is employed to model multi-modal interactions for objects referred to in the text. Second, the framework performs joint spatial localization and grasping. Extensive ablation studies have been conducted on multiple datasets to evaluate the advantages of each component in our model. Additionally, physical experiments have been performed with natural language-driven human-robot interactions on a physical robot to validate the practicality of our approach.

In the process of trajectory optimization for robot manipulator, the path that is generated may deviate from the intended path because of the adjustment of trajectory parameters, if there is limitation of end-effector path in Cartesian space for specific tasks, this phenomenon is dangerous. This paper proposes a methodology that is based on the Pareto front to address this issue, and the methodology takes into account both the multi-objective optimization of robotic arm and the quality of end-effector path. Based on dung beetle optimizer, this research proposes improved non-dominated sorting dung beetle optimizer. This paper interpolates manipulator trajectory with quintic B-spline curves, achieves multi-objective trajectory optimization that simultaneously optimizes traveling time, energy consumption, and mean jerk, proposes a trajectory selection strategy that is based on Pareto solution set by introducing the concept of Fréchet distance, and the strategy enables the end-effector to approach the desired path in Cartesian space. Simulation and experimental results validate the effectiveness and practicability of the proposed methodology on the Sawyer robot manipulator.