A pick and place robot arm with conveyor belt and vision system combines computer vision, inverse kinematics, and precise motion control into a complete industrial automation demonstration. This project is one of the most impressive engineering builds you can show at Indian college exhibitions, and it directly mirrors the systems used in electronics manufacturing facilities across Pune, Chennai, and Bengaluru. This guide covers the complete system design, vision pipeline, and Arduino/Raspberry Pi control integration.
Table of Contents
- System Architecture Overview
- Camera Vision System with OpenCV
- Robot Arm Design & IK
- Conveyor Belt & Triggering
- System Integration & Communication
- Complete Python Control Code
- Frequently Asked Questions
System Architecture Overview
A complete pick-and-place system has four subsystems working in coordination:
- Conveyor subsystem: Transports objects from an input area to the detection zone. Controlled by a DC motor with speed control.
- Vision subsystem: Camera above the detection zone identifies objects (by colour, shape, or ArUco marker) and determines their exact position for pickup.
- Robot arm subsystem: Multi-DOF arm with gripper that moves to the computed pickup position, grasps the object, and places it in the designated bin.
- Control subsystem: Raspberry Pi (running vision + IK + coordination) + Arduino (handling real-time servo/motor PWM generation).
Camera Vision System with OpenCV
The vision system uses a camera mounted at a fixed position above the conveyor. Once calibrated, pixel coordinates in the camera image can be converted to real-world robot arm coordinates.
Camera Calibration
import cv2
import numpy as np
# First, perform camera calibration using chessboard pattern
# This removes lens distortion for accurate coordinate mapping
# After calibration, define the homography:
# 4 known points in image coordinates and their real-world coordinates
image_points = np.array([
[100, 100], # Top-left corner of conveyor (pixels)
[540, 100], # Top-right
[540, 380], # Bottom-right
[100, 380], # Bottom-left
], dtype=np.float32)
real_world_points = np.array([
[0, 200], # mm from robot arm base
[300, 200],
[300, 50],
[0, 50],
], dtype=np.float32)
# Compute homography matrix (pixel -> real-world mm)
H, _ = cv2.findHomography(image_points, real_world_points)
def pixel_to_real(px, py):
"""Convert pixel coordinates to robot arm coordinates (mm)"""
pt = np.array([[[px, py]]], dtype=np.float32)
real_pt = cv2.perspectiveTransform(pt, H)
return real_pt[0][0][0], real_pt[0][0][1]Object Detection by Colour
def detect_objects(frame):
"""Detect coloured objects and return their positions"""
hsv = cv2.cvtColor(frame, cv2.COLOR_BGR2HSV)
objects = []
colour_ranges = {
'red': ([0, 100, 100], [10, 255, 255]),
'green': ([40, 100, 100], [80, 255, 255]),
'blue': ([100, 100, 100], [130, 255, 255]),
}
for colour, (lower, upper) in colour_ranges.items():
mask = cv2.inRange(hsv, np.array(lower), np.array(upper))
# Remove noise
kernel = np.ones((5,5), np.uint8)
mask = cv2.morphologyEx(mask, cv2.MORPH_CLOSE, kernel)
contours, _ = cv2.findContours(mask, cv2.RETR_EXTERNAL,
cv2.CHAIN_APPROX_SIMPLE)
for cnt in contours:
area = cv2.contourArea(cnt)
if area > 500: # Minimum object size
M = cv2.moments(cnt)
if M['m00'] > 0:
cx = int(M['m10'] / M['m00'])
cy = int(M['m01'] / M['m00'])
rx, ry = pixel_to_real(cx, cy)
objects.append({
'colour': colour,
'pixel': (cx, cy),
'real': (rx, ry),
'area': area
})
return objectsArUco Marker Detection (More Robust)
aruco_dict = cv2.aruco.getPredefinedDictionary(cv2.aruco.DICT_4X4_50)
parameters = cv2.aruco.DetectorParameters()
detector = cv2.aruco.ArucoDetector(aruco_dict, parameters)
def detect_aruco(frame):
grey = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
corners, ids, _ = detector.detectMarkers(grey)
objects = []
if ids is not None:
for i, marker_id in enumerate(ids.flatten()):
c = corners[i][0]
cx = int(np.mean(c[:, 0]))
cy = int(np.mean(c[:, 1]))
rx, ry = pixel_to_real(cx, cy)
objects.append({'id': marker_id, 'real': (rx, ry)})
return objectsRobot Arm Design & IK
For a 4-DOF pick-and-place arm:
- Joint 1 (Waist): Base rotation. Range: -90° to +90°
- Joint 2 (Shoulder): Vertical arm lift. Range: 0° to 150°
- Joint 3 (Elbow): Forearm bend. Range: 0° to 150°
- Joint 4 (Gripper): Open/close. Range: 30° to 110°
import math
# Arm dimensions
L1 = 150 # Upper arm length (mm)
L2 = 130 # Forearm length (mm)
def compute_ik(x, y, z):
"""Compute joint angles for target position (x, y, z) in mm
Returns: (theta1, theta2, theta3) in degrees, or None if unreachable
"""
# Joint 1: base rotation (waist)
theta1 = math.atan2(y, x) * 180.0 / math.pi
# Horizontal distance from base
r = math.sqrt(x**2 + y**2)
# 2D IK in (r, z) plane
dist2 = r**2 + z**2
max_reach = (L1 + L2)**2
min_reach = (L1 - L2)**2
if dist2 > max_reach or dist2 < min_reach:
return None # Unreachable
cos_t3 = (dist2 - L1**2 - L2**2) / (2 * L1 * L2)
cos_t3 = max(-1.0, min(1.0, cos_t3)) # Clamp
theta3 = math.acos(cos_t3) * 180.0 / math.pi
k1 = L1 + L2 * math.cos(math.radians(theta3))
k2 = L2 * math.sin(math.radians(theta3))
theta2 = (math.atan2(z, r) - math.atan2(k2, k1)) * 180.0 / math.pi
return theta1, theta2, theta3Conveyor Belt & Triggering
# Conveyor belt DC motor control
import RPi.GPIO as GPIO
CONVEYOR_ENABLE = 24
CONVEYOR_IN1 = 25
CONVEYOR_IN2 = 26
GPIO.setup(CONVEYOR_IN1, GPIO.OUT)
GPIO.setup(CONVEYOR_IN2, GPIO.OUT)
conveyor_pwm = GPIO.PWM(CONVEYOR_ENABLE, 1000)
conveyor_pwm.start(0)
def start_conveyor(speed=60):
GPIO.output(CONVEYOR_IN1, GPIO.HIGH)
GPIO.output(CONVEYOR_IN2, GPIO.LOW)
conveyor_pwm.ChangeDutyCycle(speed)
def stop_conveyor():
conveyor_pwm.ChangeDutyCycle(0)
# Proximity sensor (HC-SR04) to detect object arrival
def object_detected():
distance = measure_distance() # HC-SR04 measurement
return distance < 10.0 # Object within 10cm of sensorSystem Integration & Communication
The Raspberry Pi runs the vision and coordination code. It sends joint angle commands to an Arduino (or ESP32) over serial USB, which generates the precise PWM signals for each servo. This split avoids real-time timing issues on Linux.
# Raspberry Pi to Arduino serial protocol
# Command format: "A:90,B:45,C:120,G:80n"
# A=joint1, B=joint2, C=joint3, G=gripper
import serial
ser = serial.Serial('/dev/ttyUSB0', 115200, timeout=1)
def send_arm_command(j1, j2, j3, gripper):
cmd = f"A:{int(j1)},B:{int(j2)},C:{int(j3)},G:{int(gripper)}n"
ser.write(cmd.encode())
# Wait for acknowledgment
response = ser.readline().decode().strip()
return response == 'OK'Complete Python Control Code
#!/usr/bin/env python3
# main_controller.py - Pick and Place Main Loop
import cv2, time, serial, threading
import RPi.GPIO as GPIO
# Bin positions (x, y, z in mm)
BINS = {
'red': (100, -150, 50),
'green': (100, 0, 50),
'blue': (100, 150, 50),
}
HOME_POS = (150, 0, 150) # Safe home position above conveyor
def pick_and_place(obj):
colour = obj['colour']
rx, ry = obj['real']
# Move to pickup position above object
angles = compute_ik(rx, ry, 100) # 100mm above conveyor
if angles is None:
print(f"Cannot reach ({rx}, {ry}) - skipping")
return
send_arm_command(*angles, gripper=30) # Open gripper
time.sleep(0.8)
# Lower to pickup height
angles = compute_ik(rx, ry, 30) # 30mm height = near conveyor surface
send_arm_command(*angles, gripper=30)
time.sleep(0.6)
# Close gripper
send_arm_command(*angles, gripper=90) # Close
time.sleep(0.5)
# Raise
angles = compute_ik(rx, ry, 120)
send_arm_command(*angles, gripper=90)
time.sleep(0.6)
# Move to correct bin
bx, by, bz = BINS.get(colour, BINS['red'])
angles = compute_ik(bx, by, bz + 50) # Above bin
send_arm_command(*angles, gripper=90)
time.sleep(0.8)
# Lower and release
angles = compute_ik(bx, by, bz)
send_arm_command(*angles, gripper=90)
time.sleep(0.4)
send_arm_command(*angles, gripper=30) # Open gripper
time.sleep(0.4)
# Return home
home_angles = compute_ik(*HOME_POS)
send_arm_command(*home_angles, gripper=30)
print(f"Placed {colour} object in bin")
if __name__ == '__main__':
cap = cv2.VideoCapture(0)
start_conveyor(50)
while True:
ret, frame = cap.read()
if not ret: continue
if object_detected():
stop_conveyor()
time.sleep(0.3) # Let object settle
objects = detect_objects(frame)
if objects:
# Process first object
pick_and_place(objects[0])
start_conveyor(50)
cv2.imshow('Pick & Place', frame)
if cv2.waitKey(1) == ord('q'): breakFrequently Asked Questions
How accurate can a DIY pick-and-place system be?
With careful calibration, a DIY system using hobby servos can achieve ±3-5mm positional accuracy — sufficient for picking objects larger than 10mm. Industrial systems achieve sub-millimetre accuracy using servo motors with encoders, rigid frames, and temperature-compensated calibration. For better DIY accuracy: use serial bus servos with position feedback, add a camera-based correction step after initial positioning (visual servoing), and ensure the arm frame is rigid (carbon fibre or aluminium, not 3D-printed PLA).
How do I handle objects that move on the conveyor during pickup?
Calculate the conveyor speed and the time the arm takes to reach the pickup position. Predict where the object will be when the arm arrives (simple dead reckoning). Alternatively, stop the conveyor when an object is detected, wait for it to fully stop, then pick. For continuous-flow systems, use high-speed vision with tracking algorithms to compensate for object motion in real-time.
What is the maximum weight a servo-based pick-and-place arm can handle?
A well-designed 4-DOF arm with 20kg·cm shoulder servo, 15kg·cm elbow servo, and short link lengths (150mm links) can pick objects up to 300-500g reliably. Beyond 500g, servo strain and deflection become problematic. Industrial pick-and-place robots handle 1-10kg using brushless servo motors with gearboxes — a significant step up in both cost and complexity.
Add comment