Overview

For this team project, we built a system that recognizes hand gestures and uses them to control servo motors over a wireless link. We had smart factory automation in mind: a worker makes a gesture, a machine moves, and a display shows what happened.

How it works

1. Sender

A gyro sensor (MPU6050) measures acceleration and angular velocity while the user makes a gesture.
A touch sensor marks the end of the gesture.
An XBee module sends the data to the receiver.

2. Receiver

An XBee module receives the data.
The system classifies the gesture from the incoming data.
An LCD shows the recognized gesture, and the servo motors perform the matching action.

What we built

Recognition of 8 predefined gestures from the gyro sensor
Wireless XBee link between the sender and the receiver
An LCD view of the motion pattern, updated in real time from the sensor

Tools

Arduino Zero, MPU6050 gyro sensor, XBee modules, and MATLAB for preprocessing and plotting the sensor data.

Demo and documentation

Full demo

Receiver and sender

Receive

Transmit

Report

You can read our team report below or download the PDF (in Korean).

Results

The system recognized the gestures and moved the servos as intended over the wireless link, which suggested gesture control could work for simple factory tasks. Next steps would be a machine learning classifier for the gestures and detecting the end of a gesture automatically, without the touch sensor.