Robotic Arm with Vision System

2022-ECE

The objective of this project is to develop a program to provide a robotic arm with the ability to sort and stack tokens based on their assorted colors. The robotic arm needs a vision system as its primary input with a microcomputer being its primary processor. The product should be easy to learn and operate for a student without prior knowledge of robotics and computer vision. The robotic arm should have the ability to pick up objects, for example, an air-powered suction cup or an air-powered gripper. Given that the robotic arm has a pre-defined payload limit, the developed system should have the same payload limit. The cost of the product should be as low as possible while still adhering to the previously mentioned requirements.

 

Team Members

Zachary Bowen | Quinn Geist | Zheng Zeng | |

Project Poster

Click on any image to enlarge.

 

Watch the Project Video

video player icon

Download the Project Summary

video player icon

Project Summary

 

Overview

The goal of this project is to provide a robotic arm with the capability of performing pick and place operations by using a vision system. The robotic arm will be able to pick up multicolored blocks and sort them based on their colors. The system will be developed using a BeagleBone microcontroller and will be developed with the intent of teaching prospective students about the field of robotics.

Objectives

The two primary objectives to this project were to implement a computer vision system capable of detecting and classifying multicolored blocks and to perform pick and place operations with the robotic arm.

Approach

  • Each component of the system was created individually so that they could be tested before being combined with the other components.
  • The vision system uses a mask to find the colors of the tokens on the conveyor in red, blue, yellow, and green.
  • This mask is specified in the HSV color space.
  • The Dobot Magician robotic arm is calibrated before performing operation.
  • The camera is then calibrated using a calibration board.
  • After the system is calibrated, the program performs autonomously.
  • An air-powered suction cup was chosen as the end effector.
  • Blocks move down the conveyor until they are seen by a photoelectric sensor.
  • Once detected by the sensor, the camera takes an image.
  • The tokens are sorted based on the following priority: red, blue, yellow, green.
  • Tokens are able to be stacked 3 units tall, as specified by the sponsor.
  • When no blocks are detected by the sensor, the conveyor will move more blocks in front of the robot.

Outcomes

  • The sponsor will have a system that can be used for teaching interested students about the fast growing field of robotics.
  • The project is capable of performing pick/place operations with reliability
  • The robot runs on a BeagleBone microcontroller as requested by the sponsor.