Unmanned aerial vehicles (UAVs) possess fast-moving abilities and have been used in various tasks in the past decades. However, their performances are still restricted by insufficient endurance and confined environments. Intuitively, combining other locomotion modes with UAVs, such as diving and driving, would be an appropriate idea to improve the robot’s adaptability and solve the endurance problem. Recently, the terrestrial/aerial hybrid robots have drawn the researchers’ eyes for their outstanding performances, which can deploy flight mode to traverse insurmountable terrains and ground mode to increase endurance and realize detailed searches. Therefore, this paper developed the autonomous quadrotor tilting hybrid robot (AQT-HR) to achieve terrestrial/aerial dual-modal mobility and verified that the robot delivers high energy efficiency. The AQT-HR can achieve flying and driving through a quadrotor tilting mechanism, which can alter one single driving force into different directions. Furthermore, the dynamic models of the hybrid robot’s aerial and ground locomotion are derived and introduced into the model-feedforward PID control algorithm for improving the robot’s flying stability. Finally, we conducted some mobility tests and experiments about traversing obstacles to demonstrate that the proposed hybrid robot can realize autonomous mode switching and perform a low energy consumption in ground movement mode.

Most of the currently available three-degree-of-freedom manipulators are light load and cannot achieve full continuous rotation; given this, we designed a heavy-load manipulator that achieves unrestricted and continuous rotation. Due to manufacturing and assembly errors, parameter deviations between the real manipulator and its underlying theoretical model were unavoidable. Because of the lack of high-precision, high-frequency, and real-time closed-loop detection methods, we proposed a type of kinematics calibration of parameterized ant colony optimization and feedforward control methods. This was done to achieve high-precision motion control. First, an error model combining structural parameters and joint output angles was established, and the global sensitivity of each error source was analyzed to distinguish both primary and secondary sources. Based on the measured data of a laser tracker, the ant colony optimization was then used to identify six error sources. This resulted in both link length and joint driving errors of the designed manipulator. As it is a type of systematic error, the rounding error of the theoretical trajectory was carefully analyzed, and feedforward control methods with different coefficients were designed to further improve positioning accuracy based on the kinematic calibration. Experimental results showed that the proposed kinematic calibration and feedforward control methods achieved relatively precise motion control for the designed manipulator.

Robots with multi-sensors always have a problem of weak pairing among different modals of the collected information produced by multi-sensors, which leads to a bad perception performance during robot interaction. To solve this problem, this paper proposes a Force Vision Sight (FVSight) sensor, which utilizes a distributed flexible tactile sensing array integrated with a vision unit. This innovative approach aims to enhance the overall perceptual capabilities for object recognition. The core idea is using one perceptual layer to trigger both tactile images and force-tactile arrays. It allows the two heterogeneous tactile modal information to be consistent in the temporal and spatial dimensions, thus solving the problem of weak pairing between visual and tactile data. Two experiments are specially designed, namely object classification and slip detection. A dataset containing 27 objects with deep presses and shallow presses is collected for classification, and then 20 slip experiments on three objects are conducted. The determination of slip and stationary state is accurately obtained by covariance operation on the tactile data. The experimental results show the reliability of generated multimodal data and the effectiveness of our proposed FVSight sensor.

The language-guided visual robotic grasping task focuses on enabling robots to grasp objects based on human language instructions. However, real-world human-robot collaboration tasks often involve situations with ambiguous language instructions and complex scenarios. These challenges arise in the understanding of linguistic queries, discrimination of key concepts in visual and language information, and generation of executable grasping configurations for the robot’s end-effector. To overcome these challenges, we propose a novel multi-modal transformer-based framework in this study, which assists robots in localizing spatial interactions of objects using text queries and visual sensing. This framework facilitates object grasping in accordance with human instructions. Our developed framework consists of two main components. First, a visual-linguistic transformer encoder is employed to model multi-modal interactions for objects referred to in the text. Second, the framework performs joint spatial localization and grasping. Extensive ablation studies have been conducted on multiple datasets to evaluate the advantages of each component in our model. Additionally, physical experiments have been performed with natural language-driven human-robot interactions on a physical robot to validate the practicality of our approach.

In the process of trajectory optimization for robot manipulator, the path that is generated may deviate from the intended path because of the adjustment of trajectory parameters, if there is limitation of end-effector path in Cartesian space for specific tasks, this phenomenon is dangerous. This paper proposes a methodology that is based on the Pareto front to address this issue, and the methodology takes into account both the multi-objective optimization of robotic arm and the quality of end-effector path. Based on dung beetle optimizer, this research proposes improved non-dominated sorting dung beetle optimizer. This paper interpolates manipulator trajectory with quintic B-spline curves, achieves multi-objective trajectory optimization that simultaneously optimizes traveling time, energy consumption, and mean jerk, proposes a trajectory selection strategy that is based on Pareto solution set by introducing the concept of Fréchet distance, and the strategy enables the end-effector to approach the desired path in Cartesian space. Simulation and experimental results validate the effectiveness and practicability of the proposed methodology on the Sawyer robot manipulator.

SLAM Benchmark plays a pivotal role in the field by providing a common ground for performance evaluation. In this paper, a novel methodology of simultaneous localization and mapping benchmark and map accuracy improvement (SLAMB&MAI) is introduced. It can objectively evaluate errors of localization and mapping, and further improve map accuracy by utilizing evaluation results as feedback. The proposed benchmark transforms all elements into a global frame and measures the errors between them. The comprehensiveness consists in the benchmark of both localization and mapping, and the objectivity consists in the consideration of the correlation between localization and mapping by the preservation of the original pose relations between all reference frames. The map accuracy improvement is realized by first obtaining the optimization that minimizes the errors between the estimated trajectory and ground truth trajectory and then applying it to the estimated map. The experimental results showed that the map accuracy can be improved by an average of 15%. The optimization that yields minimal localization errors is obtained by the proposed Centre Point Registration-Iterative Closest Point (CPR-ICP). This proposed Iterative Closest Point (ICP) variant pre-aligns two point clouds by their centroids and least square planes and then uses traditional ICP to minimize the error between them. The experimental results showed that CPR-ICP outperformed traditional ICP, especially in cases involving large-scale environments. To the extent of our knowledge, this is the first work that can not only objectively benchmark both localization and mapping but also revise the estimated map and increase its accuracy, which provides insights into the acquisition of ground truth map and robot navigation.

Visual simultaneous localisation and mapping (vSLAM) has shown considerable promise in positioning and navigating across a variety of indoor and outdoor settings, significantly enhancing the mobility of robots employed in industrial and everyday services. Nonetheless, the prevalent reliance of vSLAM technology on the assumption of static environments has led to suboptimal performance in practical implementations, particularly in unstructured and dynamically noisy environments such as substations. Despite advancements in mitigating the influence of dynamic objects through the integration of geometric and semantic information, existing approaches have struggled to strike an equilibrium between performance and real-time responsiveness. This study introduces a lightweight, multi-modal semantic framework predicated on vSLAM, designed to enable intelligent robots to adeptly navigate the dynamic environments characteristic of substations. The framework notably enhances vSLAM performance by mitigating the impact of dynamic objects through a synergistic combination of object detection and instance segmentation techniques. Initially, an enhanced lightweight instance segmentation network is deployed to ensure both the real-time responsiveness and accuracy of the algorithm. Subsequently, the algorithm’s performance is further refined by amalgamating the outcomes of detection and segmentation processes. With a commitment to maximising performance, the framework also ensures the algorithm’s real-time capability. Assessments conducted on public datasets and through empirical experiments have demonstrated that the proposed method markedly improves both the accuracy and real-time performance of vSLAM in dynamic environments.

In order to improve the positioning accuracy of industrial robots, this paper proposes a global calibration method for planar parallel robot considering joint errors, which solves the problem that the existing calibration methods only consider part of the error sources and the calibration accuracy is poor, and improves the calibration efficiency and robot positioning accuracy. Consequently, it improves calibration efficiency and the overall precision of robot positioning. Firstly, the error model of overdetermined equations combined with structural parameters is established, and the global sensitivity of each error source is analyzed. Based on the measurement data of laser tracker, the local error source is identified by the least square method, which improves the local error accuracy by 88.6%. Then, a global error spatial interpolation method based on inverse distance weighting method is proposed, and the global accuracy is improved by 59.16%. Finally, the radial basis function neural network error prediction model with strong nonlinear approximation function is designed for global calibration, and the accuracy is improved by 63.05%. Experimental results verify the effectiveness of the proposed method. This study not only provides technical support for the engineering application of this experimental platform but also provides theoretical guidance for the improvement of the accuracy of related robot platforms.

In the current investigation, a two-stage hybridization model has been used for the motion planning of humanoids in complex environmental conditions using regression analysis and the firefly algorithm. In the first step, sensory outputs are fed to the regression model, and an initial turning angle (ITA) is obtained. In the second step, the ITA is again fed as input to the firefly model along with other required inputs, and the final turning angle (FTA) is obtained. The FTA serves as the guiding parameter for the humanoids to reach their desired destinations. The developed motion planning scheme has been implemented on NAO humanoid robots in simulation and experimental platforms. A Petri-Net control strategy has been integrated along with the hybrid scheme while negotiating multiple humanoids in a common platform. The results obtained from the motion planning analysis in simulation and experimental arenas are compared against each other in terms of selected navigational parameters and observed satisfactory agreements. Finally, the proposed hybrid controller is also tested against another standard navigational model and substantial enhancement in the performance has been noticed.

This paper systematically investigates a soft-rigid hybrid pneumatic actuator (SRHPA), which consists of a rigid-foldable twisting skeleton capable of a large range of helical motion and a soft bellows muscle with high linear driving force. Considering the unique varying-pitch helical motion of the foldable skeleton, the analytical model mapping the input force generated by the bellows muscle and output forces of the actuator is revealed and verified with a simulation of the force analysis. Prototypes of the actuator are developed by fabricating the twisting skeleton with multilayered aluminum composite panels and 3D-printing the bellows muscle with thermoplastic polyurethane (TPU) 95A filament. The static and dynamic performances of the prototypes are tested to validate the analytical modeling of output forces. Using the actuator as a module, a novel bipedal inchworm robot with four modules is developed and tested to demonstrate its adaptability in confined space by switching between the going-straight, the turning-around, and the rotating gaits. The hybrid actuator and the inchworm robot with zero onboard electronics have the potential to be deployed in extreme environments where pneumatically actuated systems are preferred over electrical machines and drives, such as in nuclear and explosive environments.

The increase of human presence in the subsea and seabed environments necessitates the development of more capable and highly dexterous, innovative underwater manipulators. Biomimetic soft-robot arms represent a promising candidate for such manipulation systems. However, the well-known modeling techniques and control theories of traditional rigid robots do not apply to soft robots. The challenges of kinematic and dynamic modeling of soft robots with infinite degrees of freedom require the development of dedicated modeling methods. A novel procedure for representing soft-robotic arms and their motion in a rigid-body simulation environment is proposed in this paper. The proposed procedure relies on the piecewise constant curvature approach to simplify the very complex model of hyper-redundant soft-robotic arms, making it suitable for real-time applications. The proposed method is implemented and verified to be used in model-mediated teleoperation of the soft arms of a biomimetic robotic squid designed for underwater manipulation as a case study.

In this study, we present a novel approach to generate limit cycle walking using nonlinear model predictive control (NMPC). Output-zeroing control is now widely used as a control method to limit cycle walking. This control offers strong feedback to the desired trajectory and the generation of energy-efficient and robust limit cycle walking. However, we observed that this method disables the natural dynamics of the robot, leading to problems regarding energy efficiency during walking. This study demonstrates that the energy consumption of walking using the output-zeroing control increases significantly in a disturbed environment. To overcome this limitation, the proposed approach leverages the robot’s dynamics using NMPC to achieve energy-efficient walking even in a disturbed environment. We demonstrate the practicality of the proposed method using two different simulation environments.

In this paper, an $\textrm{H}_{{\infty }}$ dynamic output feedback controller is experimentally implemented for the position regulation of a fully actuated tilted-rotor octocopter unmanned aerial vehicle (UAV) to improve wind disturbance rejection during station-keeping. To apply the lateral forces, besides the standard tilt-to-translate (attitude-thrust) movement, tilted-rotor UAVs can generate vectored (horizontal) thrust. Vectored-thrust is high-bandwidth but saturation-constrained, while attitude-thrust generates larger forces with lower bandwidth. For the first time, this paper emphasizes the frequency-dependent allocation of weighting matrices in $\textrm{H}_{{\infty }}$ control design based on the physical capabilities of the fully actuated UAV (vectored-thrust and attitude-thrust). A dynamic model of the tilted-rotor octocopter, including aerodynamic effects and rotor dynamics, is presented to design the controller. The proposed $\textrm{H}_{{\infty }}$ controller solves the frequency-dependent actuator allocation problem by augmenting the dynamic model with weighting transfer functions. This novel frequency-dependent allocation utilizes the attitude-thrust for low-frequency disturbances and vectored-thrust for high-frequency disturbances, which exploits the maximum potential of the fully actuated UAV. Several wind tunnel experiments are conducted to validate the model and wind disturbance rejection performance, and the results are compared to the baseline PX4 Autopilot controller on both the tilted-rotor and a planar octocopter. The $\textrm{H}_{{\infty }}$controller is shown to reduce station-keeping error by up to 50% for an actuator usage 25% higher in free-flight tests.

This paper presents a comprehensive strategy to improve the locomotion performance of humanoid robots on various slippery floors. The strategy involves the implementation and adaptation of a divergent component of motion (DCM) based control architecture for the humanoid NAO, and the introduction of an embedded yaw controller (EYC), which is based on a proportional-integral-derivative (PID) control algorithm. The EYC is designed not only to address the slip behavior of the robot on low-friction floors but also to tackle the issue of non-straight walking patterns that we observed in this humanoid, even on non-slippery floors. To fine-tune the PID gains for the EYC, a systematic trial-and-error approach is employed. We iteratively adjusted the P (Proportional), I (Integral), and D (Derivative) parameters while keeping the others fixed. This process allowed us to optimize the PID controller’s response to different walking conditions and floor types. A series of locomotion experiments are conducted in a simulated environment, where the humanoid step frequency and PID gains are varied for each type of floor. The effectiveness of the strategy is evaluated using metrics such as robot stability, energy consumption, and task duration. The results of the study demonstrate that the proposed approach significantly improves humanoid locomotion on different slippery floors, by enhancing stability and reducing energy consumption. The study has practical implications for designing more versatile and effective solutions for humanoid locomotion on challenging surfaces and highlights the adaptability of the existing controller for different humanoid robots.

In this paper, a method of planning the expanded S-curve trajectory of robotic manipulators is proposed to minimize the execution time as well as to achieve the smoother trajectory generation in the deceleration stage for point-to-point motions. An asymmetric parameter is added to the piecewise sigmoid function for an improved jerk profile. This asymmetric profile is continuous and infinitely differentiable. Based on this profile, two analytical algorithms are presented. One is applied to determine the suitable time intervals of trajectory satisfying the time optimality under the kinematic constraints, and the other is to determine the asymmetric parameter generating the minimum execution time. Also, the calculation procedure for the time-scaled synchronization for all joints is given to decrease unnecessary loads onto the actuators. The velocity, acceleration, jerk and snap (the derivative of jerk) of the joints and the end-effector are equal to zero at two end points of motion. The simulation results through 3 DOF and 6 DOF robotic manipulators show that our approach reduces the jerk and snap of the deceleration stage effectively while decreasing the total execution time. Also, the analysis for a single DOF mass-spring-damper system indicates that the residual vibration could be reduced to 10% more than the benchmark techniques in case velocity, acceleration and jerk are limited to 1.24 m/s, 6 m/s2 and 80 m/s3, respectively and displacement is set to 0.8m. These results manifest that the performance of reducing residual vibrations is good and demonstrate an important characteristic of the proposed profile suitable for point-to-point motion.

Visual odometry (VO) is a key technology for estimating camera motion from captured images. In this paper, we propose a novel RGB-D visual odometry by constructing and matching features at the superpixel level that represents better adaptability in different environments than state-of-the-art solutions. Superpixels are content-sensitive and perform well in information aggregation. They could thus characterize the complexity of the environment. Firstly, we designed the superpixel-based feature SegPatch and its corresponding 3D representation MapPatch. By using the neighboring information, SegPatch robustly represents its distinctiveness in various environments with different texture densities. Due to the inclusion of depth measurement, the MapPatch constructs the scene structurally. Then, the distance between SegPatches is defined to characterize the regional similarity. We use the graph search method in scale space for searching and matching. As a result, the accuracy and efficiency of matching process are improved. Additionally, we minimize the reprojection error between the matched SegPatches and estimate camera poses through all these correspondences. Our proposed VO is evaluated on the TUM dataset both quantitatively and qualitatively, showing good balance to adapt to the environment under different realistic conditions.

Reinforcement learning (RL) has been successfully applied to a wealth of robot manipulation tasks and continuous control problems. However, it is still limited to industrial applications and suffers from three major challenges: sample inefficiency, real data collection, and the gap between simulator and reality. In this paper, we focus on the practical application of RL for robot assembly in the real world. We apply enlightenment learning to improve the proximal policy optimization, an on-policy model-free actor-critic reinforcement learning algorithm, to train an agent in Cartesian space using the proprioceptive information. We introduce enlightenment learning incorporated via pretraining, which is beneficial to reduce the cost of policy training and improve the effectiveness of the policy. A human-like assembly trajectory is generated through a two-step method with segmenting objects by locations and iterative closest point for pretraining. We also design a sim-to-real controller to correct the error while transferring to reality. We set up the environment in the MuJoCo simulator and demonstrated the proposed method on the recently established The National Institute of Standards and Technology (NIST) gear assembly benchmark. The paper introduces a unique framework that enables a robot to learn assembly tasks efficiently using limited real-world samples by leveraging simulations and visual demonstrations. The comparative experiment results indicate that our approach surpasses other baseline methods in terms of training speed, success rate, and efficiency.

The manufacturing of the X-shaped tip of prestressed centrifugal concrete piles is nowadays done half automatically by combining the manual worker and the automatic welding robot. To make this welding process full automatically, the welding seam tracking algorithm is considered. There are many types of sensors that can be used to detect the welding seam such as vision sensor, laser vision sensor, arc sensor, or touch sensor. Each type of sensor has its advantages and disadvantages. In this paper, an algorithm for welding seam tracking using laser distance sensor is proposed. Firstly, the fundamental mathematics theory of the algorithm is presented. Next, the positioning table system supports the procedure is designed and manufactured. The object of this research is the fillet joint because of the characteristics of the X-shaped tip of the concrete piles. This paper proposes a new method to determine the welding trajectory of the tip using laser distance sensor. After that, the experimental results are received to verify the proposed idea. Finally, the improved proposal of the algorithm is considered to increase the accuracy of the suggested algorithm.

Bag manipulation through robots is complex and challenging due to the deformability of the bag. Based on the dynamic manipulation strategy, we propose a new framework, ShakingBot, for the bagging tasks. ShakingBot utilizes a perception module to identify the key region of the plastic bag from arbitrary initial configurations. According to the segmentation, ShakingBot iteratively executes a novel set of actions, including Bag Adjustment, Dual-arm Shaking, and One-arm Holding, to open the bag. The dynamic action, Dual-arm Shaking, can effectively open the bag without the need to take into account the crumpled configuration. Then, the robot inserts the items and lifts the bag for transport. We perform our method on a dual-arm robot and achieve a success rate of 21/33 for inserting at least one item across various initial bag configurations. In this work, we demonstrate the performance of dynamic shaking action compared to the quasi-static manipulation in the bagging task. We also show that our method generalizes to variations despite the bag’s size, pattern, and color. Supplementary material is available at https://github.com/zhangxiaozhier/ShakingBot.

In practical applications, many robots equipped with embedded devices have limited computing capabilities. These limitations often hinder the performance of existing dynamic SLAM algorithms, especially when faced with occlusions or processor constraints. Such challenges lead to subpar positioning accuracy and efficiency. This paper introduces a novel lightweight dynamic SLAM algorithm designed primarily to mitigate the interference caused by moving object occlusions. Our proposed approach combines a deep learning object detection algorithm with a Kalman filter. This combination offers prior information about dynamic objects for each SLAM algorithm frame. Leveraging geometric techniques like RANSAC and the epipolar constraint, our method filters out dynamic feature points, focuses on static feature points for pose determination, and enhances the SLAM algorithm’s robustness in dynamic environments. We conducted experimental validations on the TUM public dataset, which demonstrated that our approach elevates positioning accuracy by approximately 54% and boosts the running speed by 75.47% in dynamic scenes.