Introduction: In industrial applications, robots are commonly used to perform material handling tasks. However, parameters such as the color, shape, size, and orientation of the material are beyond the robot's capability to detect independently. Vision cameras, on the other hand, can effectively address this requirement. When exchanging data between the two systems, programming personnel need to standardize or extract the relevant information. Today, we will discuss the communication and data processing between an ABB robot and a vision system. For this setup, we utilize Socket communication between the robot and the vision system. We will not delve into the details of this protocol here; instead, the focus will be on explaining the process of data extraction and conversion between the two systems. As shown in Figure 1, the robot intends to pick up a material, but the orientation of the material can only be provided to the robot by the vision camera (a 2D camera capable of recognizing planar rotation angles).
Before programming, we must understand that for a robot to accurately pick up a material, two conditions must be met: 1) the position values of the material, and 2) the rotation angle value of the material. Therefore, the vision system needs to provide three data points: the X and Y position data of the material and the Z rotation angle value. However, the rotational posture of an ABB robot's tool is determined by four parameters—q1, q2, q3, q4 (quaternions)—while the value sent by the vision camera is a rotation angle (Euler angle). Thus, it is necessary to convert the Euler angle into quaternions so that the robot can adjust its tool posture appropriately to pick up the material.
MODULE Module1
VAR num cam_x := 0; ! Define X position data sent by vision
VAR num cam_y := 0; ! Define Y position data sent by vision
VAR num cam_angle := 0; ! Define rotation angle data sent by vision
CONST robtarget p10 := [[-1.289131E-05, 3.351449E-06, 237.1972], [0.01633704, -3.806521E-08, 0.9998665, -3.595199E-08], [-1, 0, -1, 0], [9E+09, 9E+09, 9E+09, 9E+09, 9E+09, 9E+09]]; ! Define safe position data
VAR robtarget ppick := [[24.6736, 49.99907, 19.99434], [0.01633701, -5.075877E-09, 0.9998665, -2.882863E-08], [-1, 0, -1, 0], [9E+09, 9E+09, 9E+09, 9E+09, 9E+09, 9E+09]]; ! Define pick position data
VAR socketdev Socket1; ! Define socket data
VAR string received_string; ! Define string data
VAR num startBit1; ! Define num data startBit1
VAR num startBit2; ! Define num data startBit2
VAR num startBit3; ! Define num data startBit3
VAR num startBit4; ! Define num data startBit4
VAR num startBit5; ! Define num data startBit5
VAR num EndBit1; ! Define num data
VAR num EndBit2; ! Define num data
VAR num EndBit3; ! Define num data
VAR num LenBit1; ! Define num data
VAR num LenBit2; ! Define num data
VAR num LenBit3; ! Define num data
VAR num Lenstring; ! Define num data
VAR string XData := ""; ! Define string data
VAR string yData := ""; ! Define string data
VAR string AngleData := ""; ! Define string data
PERS num X; ! Define persistent num data
PERS num Y; ! Define persistent num data
PERS num Angle; ! Define persistent num data
PROC main()
SocketCreate Socket1; ! Create socket
SocketConnect Socket1, "127.0.0.1", 3000; ! Connect socket
WHILE TRUE DO
SocketSend Socket1 \Str := "hello server"; ! Send data to vision camera
SocketReceive Socket1 \Str := received_string; ! Receive data to robot
TPWrite "server wrote-" + received_string; ! Display on screen
DecodeData; ! Data parsing procedure
cal_data; ! Euler angle to quaternion conversion procedure
r_move; ! Pick procedure
ENDWHILE
ERROR
SocketClose Socket1; ! Close communication on error
ENDPROC
PROC DecodeData() ! Data conversion procedure
VAR string Strread := ""; ! Define string data
VAR bool datatrue := FALSE; ! Define boolean data
Strread := received_string; ! Store received data in Strread
Lenstring := StrLen(Strread); ! Determine length of received data
startBit1 := 1;
EndBit1 := StrFind(Strread, startBit1, ";"); ! Find first semicolon
LenBit1 := EndBit1 - startBit1; ! Determine length of first data segment
startBit2 := EndBit1 + 1;
EndBit2 := StrFind(Strread, startBit2, ";"); ! Find second semicolon
LenBit2 := EndBit2 - startBit2; ! Determine length of second data segment
startBit3 := EndBit2 + 1;
EndBit3 := StrFind(Strread, startBit3, ";"); ! Find third semicolon
LenBit3 := EndBit3 - startBit3; ! Determine length of third data segment
XData := StrPart(Strread, startBit1, LenBit1); ! Extract first data (X)
yData := StrPart(Strread, startBit2, LenBit2); ! Extract second data (Y)
AngleData := StrPart(Strread, startBit3, LenBit3); ! Extract third data (Angle)
datatrue := StrToVal(XData, cam_x); ! Convert string to num
datatrue := StrToVal(yData, cam_y); ! Convert string to num
datatrue := StrToVal(AngleData, cam_angle); ! Convert string to num
ENDPROC
PROC cal_data() ! Euler angle to quaternion conversion procedure
VAR num rx; ! Define num data
VAR num ry; ! Define num data
VAR num rz; ! Define num data
ppick.trans.z := 0; ! 2D vision planar rotation, Z set to 0
rx := EulerZYX(\X, ppick.rot); ! Convert quaternion to Euler angle X
ry := EulerZYX(\Y, ppick.rot); ! Convert quaternion to Euler angle Y
rz := EulerZYX(\Z, ppick.rot); ! Convert quaternion to Euler angle Z
ppick.trans.x := cam_x; ! Assign camera X position to pick point X
ppick.trans.y := cam_y; ! Assign camera Y position to pick point Y
ppick.rot := OrientZYX(cam_angle, ry, rx); ! Convert Euler angles to quaternion
ENDPROC
PROC r_move() ! Pick procedure
MoveL Offs(ppick, 0, 0, 30), v1000, z50, xipan \WObj := Workobject_2; ! Move above pick point
MoveL ppick, v1000, fine, xipan \WObj := Workobject_2; ! Move to pick point
WaitTime 0.5; ! Wait 0.5 seconds
MoveL Offs(ppick, 0, 0, 30), v1000, fine, xipan \WObj := Workobject_2; ! Move above pick point
MoveL p10, v1000, fine, xipan \WObj := Workobject_2; ! Return to safe point
ENDPROC
ENDMODULE- Which data does the vision system send to the robot, and in what format is it transmitted?
- How is the data sent by the vision system extracted by the robot?
- How are the Euler angles from the vision system converted into the robot’s tool rotation angles?
If a vision system is unavailable, debugging software can be used for testing.
