P-03prototypeAI & Embedded Systems · 2025
SIBI Hand Gesture Recognition and LED System Control Using Deep Neural Network
A real-time SIBI hand-gesture recognition system built with MediaPipe and a deep neural network, integrated with Arduino for LED and display control.
- Role
- Developer · ML, Vision & Hardware Integration
- Timeframe
- Jun 2025 — Jun 2025
- Domain
- AI & Embedded Systems
- Status
- prototype
- Accuracy
- >90%
- Input
- 3D Hand Landmarks
- Interface
- Tkinter + Serial
Overview
This project bridges accessibility and embedded intelligence by recognizing Bahasa Isyarat Indonesia (SIBI) gestures in real time using MediaPipe and a deep neural network.
The pipeline covers dataset collection, model training, live prediction, and hardware feedback through an Arduino Uno connected over serial.
The Challenge
The main challenge was turning noisy real-time hand landmarks into stable gesture predictions while keeping the system responsive enough for live use and hardware control.
Approach
- 01Built a custom dataset collector using OpenCV and MediaPipe to record structured landmark samples.
- 02Trained a multi-layer DNN classifier on 3D hand landmark data for gesture prediction.
- 03Created a Tkinter dashboard for live visualization, confidence feedback, and word-buffer management.
- 04Linked predictions to Arduino-based outputs so gestures can trigger LED or OLED responses.
Outcomes
- Delivered a real-time gesture recognition pipeline with live confidence feedback.
- Reached over 90% model accuracy on the trained landmark dataset.
- Enabled practical hardware control from recognized SIBI gestures.
Stack
Vision
- OpenCV
- MediaPipe
- Hand Landmark Tracking
ML
- Python
- Deep Neural Network
- NumPy
Hardware
- Arduino Uno
- Serial Communication
- LED / OLED Output