Mingrui YU

Mingrui YU

A robotics learner

Projects

ICRA 2024 Robotic Grasping and Manipulation Competition - In-Hand Manipulation Track

The Robotic Grasping and Manipulation Competition - In-Hand Manipulation Track is focused on reconfiguring objects in robot dexterous hands. The competition tasks focus on two essential skills: in-hand precise manipulation and in-hand object re-orientation. Our solution involving an open-sourced Leap Hand, trajectory optimization, and reinforcement learning won the 1st place of the in-hand manipulation competition, and also won the Most Elegant Solution Award among all tracks in the RGMC.

In-Hand Following of Deformable Linear Objects Using Dexterous Fingers With Tactile Sensing

Most research on deformable linear object (DLO) manipulation assumes rigid grasping. However, beyond rigid grasping and re-grasping, in-hand following is also an essential skill that humans use to dexterously manipulate DLOs. In this work, inspired by how humans use fingers to follow DLOs, we explore the usage of a generic dexterous hand with tactile sensing to imitate human skills and achieve robust in-hand DLO following. To enable the hardware system to function in the real world, we develop a framework that includes Cartesian-space arm-hand control, tactile-based in-hand 3-D DLO pose estimation, and task-specific motion design.

[Website]

Generalizable Global Manipulation of Deformable Linear Objects in Constrained Environments

This article focuses on the global moving and shaping of DLOs in constrained environments by dual-arm robots. The main objectives are 1) to efficiently and accurately accomplish this task, and 2) to achieve generalizable and robust manipulation of various DLOs. To this end, we propose a complementary framework with whole-body planning and control using appropriate DLO model representations. Experiments demonstrate that our framework can accomplish considerably more complicated tasks than existing works. It achieves a 100% planning success rate among thousands of trials with an average time cost of less than 15 second, and a 100% manipulation success rate among 135 real-world tests on five different DLOs.

[Website]

Dual-Arm Manipulation of Deformable Linear Objects with Whole-Body Obstacle Avoidance

Manipulating deformable linear objects (DLOs) to achieve desired shapes in constrained environments with obstacles is a meaningful but challenging task. We propose a coarse-to-fine framework to combine global planning and local control for dual-arm manipulation of DLOs, capable of precisely achieving desired configurations and avoiding potential collisions between the DLO, robot, and obstacles. Both simulations and real-world experiments demonstrate that our framework can robustly achieve desired DLO configurations in constrained environments with imprecise DLO models, which may not be reliably achieved by only planning or control.

[ICRA Website]

Learning to Estimate 3-D States of Deformable Linear Objects from Occluded Single-Frame Point Clouds

We focus on learning to robustly estimate the states of DLOs from single-frame point clouds in the presence of occlusions using a data-driven method. Simulation and real-world experimental results demonstrate that our method can generate globally smooth and locally precise DLO state estimation results even with heavily occluded point clouds, which can be directly applied to real-world robotic manipulation of DLOs in 3-D space.

Global Model Learning for Large Deformation Control of Deformable Linear Objects

We propose a coupled offline and online data-driven method for efficiently learning a global deformation model, allowing for both accurate modeling through offline learning and further updating for new DLOs via online adaptation. We also propose a convex-optimization-based controller and analyze the system’s stability using the Lyapunov method. Detailed simulations and real-world experiments demonstrate that our method can efficiently and precisely estimate the deformation model, and achieve large deformation control of untrained DLOs in 2D and 3D dual-arm manipulation tasks better than the existing methods. learning.

[T-RO Website] [ICRA Website]

ICRA2022 RoboMaster University Sim2Real Challenge

The RoboMaster University Sim2Real Challenge at ICRA 2022 aims to optimize the system performance of a robotic task which consists of perception, navigation and manipulation. The first stage is the simulation stage, in which teams will fus on developing algorithms in a simulated environment and completing the challenge mission. In the simulation stage, our team ranked 4th among all 117 participating teams (according to the task performance).

Target Capture of Space Robot

In this course project, we propose a scheme for capturing a target satellite out of control using a space robot. The space robot is a service satellite with two robot manipulators. During the on-orbit service, the space robot needs to capture the disabled and spinning target satellite, stop its spinning, restabilize it, and fix it up. We design the robot mechanics and the gripper mechanics, analyze the workspace of the manipulators, and simulate the target capture process based on the PyBullet simulator.

[课程报告] [Code]

Seq-CALC: lightweight and robust deep loop detection for SLAM

We propose a lightweight SLAM loop detection method based on deep learning. An unsupervised denoising auto-encoder network is trained to extract an image’s deep descriptor, which uses the projective-transformed images as input data, and HOG descriptor as the reconstruction object. Besides, we apply PCA to reduce the descriptor’s dimension, and combine linear query with fast approximate nearest neighbor search to further improve the efficiency. With the help of sequence match, we significantly improve its accurary. The results of experiments on NVIDIA TX2 demonstrate that, our Seq-CALC outperforms DBoW3 in terms of both accurary and efficiency on various challenging datasets. Its accuary is even close to NetVLAD under certain conditions. We open-source the C++ library of Seq-CALC, which can be conveniently embedded in any SLAM system.

[Details] [Code] [本科毕业论文]

A simple stereo SLAM system

This is a simple stereo SLAM system with a deep-learning based loop closure module (Seq-CALC). As a beginner of SLAM, I made this system mainly in order to practice my coding and engineering skills to build a full SLAM system by myself. The structure of the system is simple and clear, in which I didn’t apply much detailed optimization. Thus, the performance of this system is not outstanding. Such a simple structure may be friendly for a SLAM beginner to study the body frame of a full SLAM system.

[Details] [Code]

An Intelligent Car based on Raspberry Pi

In this curriculum project, we utilize Raspberry Pi 3 and some other accessories to design a simple intelligent car. The functions we have implemented include obstacle avoidance, video transmittion from Pi to PC, lane tracking, object detection, and tennis tracking.

[Details] [Code]

Adaptive traffic signal control with deep reinforcement learning

Contents:

[Paper]

Image processing in color+depth based 3D vision systems

Contents: