Main Menu

Lecture Review: Robotic Manipulation of Deformable Objects in 3C Manufacturing

  • 2020.07.26
  • News
On July 22, AIRS invited Professor Xiang Li from the Department of Automation, Tsinghua University to give a lecture on Robotic Manipulation of Deformable Objects in 3C Manufacturing. Here is a review of the lecture

On July 22, Prof. LI Xiang from the Department of Automation of Tsinghua University to brought us a live lecture on "Robotic Manipulation of Deformable Objects in 3C Manufacturing". The lecture was hosted by Prof. SUN Zhenglong from The Chinese University of Hong Kong, Shenzhen.

At the beginning of the talk, Prof. Li first introduced his research experiences and prospects of these studies. From cell manipulation to exoskeleton, from surgical to industrial applications. In all these research topics, Prof. Li focused on dealing with robotic manipulation in an unstructured environment, with uncertainties, and unknown model.

(Figure 1 Application background of visual control operation)

In terms of application, Prof. Li showed that there is great demand in 3C (computer, communication and consumer electronics) manufacturing to use robots to replace human labor, for example the manufacturing and assembly of flexible PCBs and USB wires. These tasks could be labor intensive, and long-time operation may lead to fatigue and safety issues. But the involvement of deformable objects makes it difficult to be solved with existing robotic manipulation solutions. As a result, close physical contact between the robots and the objects must be guaranteed all the time, and only low manipulation speed is applied for stability concern. Prof. Li aims to tackle these limitations, and has made contributions to the feasibility, degree of automation and stability of robots in contact with deformable objects.

(Figure 2 Application Prospect of Vision Control Operation)

Subsequently, Prof. Li used flexible PCB welding as an example to explain vision-based robotic manipulation. Through visual parameter collection of the environment, combined with depth, deformation model, camera model and Jacobian matrix of forward kinematics, the joint velocity is converted into characteristic speed to plan the path. But the deformation model is usually unknown, and difficult to be modeled.

(Figure 3 Schematic diagram of the core design of the welding PCB robot)

Based on this, Prof. Li showed an overall control diagram for the visual-based robotic manipulation of flexible PCB. The proposed controller enables the robot to automatically contact then actively deform the flexible PCB into a desired configuration, such that the well deformed PCB can be soldered in an automatic soldering machine. Both operations of contacting and active deformation are smoothly integrated into a single continuous controller and automatically transited by assessing the physical contact online. Prof. Li also explained the adaptive Neural-network techniques employed to approximate the unknown deformation model and the uncalibrated camera model. The dynamic stability of the closed-loop system can be further rigorously proved with Lyapunov methods, and experimental results are presented to illustrate the performance of the proposed controller.

(Figure 4 Block diagram of the control strategy for vision-based flexible PCB welding robots)

Next, Prof. Li introduced another vision-based manipulation example, automated assembly of USB wires. It requires sorting the four colors (red, white, green and black) in order and grasp them and put in the holder for machine soldering. A two-level hierarchical control strategy is developed and embedded into the controller, where Level-I is referred to as the grasping and manipulation of wires, and Level-II is referred to as the wire alignment by following the USB color code. The proposed controller allows the robot to automatically grasp, manipulate, and align the wires. Two types of experimental result were demonstrated for this application.

(Figure 5 USB wires assembly robot based on visual control principle diagram)

(Figure 6 The deformable linear object is based on the control principle diagram of the visual robot)

Lastly, Prof. Li introduced a project on the vision-based robotic manipulation on linear deformable objects. Manipulating a linear deformable object (DLO, e.g. wires, cables, ropes) into the desired shape is very challenging for robots, because the DLO exhibits high degrees of freedom (DOFs), and multiple feature points along the DLO are closely coupled. In order to solve related problems, a model-based control scheme for robot manipulators is proposed. The shape of DLO is represented with multiple feature points, and the desired shape is specified as a series of dynamic regions where a feature point can move freely inside the corresponding region to suit the movement of other points. Then, multiple feature points are manipulated in a sequential manner such that the controllable inputs are always more than or equal to the error outputs. Prof. Li explained related control principles and demonstrated experiments, which was very interesting and inspiring.


At the end of the speech, Prof. Li had a friendly interaction with the audience in the Q&A session. Prof. Li also gave his insights of potential applications of the vision-based robotic manipulation technology.

Video Archive