🖱️ Gesture Controlled Virtual Mouse

An AI-powered, touchless interface that uses hand gestures and voice commands to control the computer—no physical contact required.

🔍 Description

This project simplifies human-computer interaction by utilizing state-of-the-art Machine Learning and Computer Vision to detect static and dynamic hand gestures using MediaPipe and CNN. It also includes a voice assistant named Proton to handle system operations via speech, all without requiring any special hardware. Compatible with Windows systems.

✨ Features

🎯 Gesture Recognition:

• Move Cursor
• Left Click, Right Click, Double Click
• Scroll, Drag & Drop
• Multiple Item Selection
• Volume & Brightness Control

🎙️ Voice Assistant – Proton:

• Start/Stop Gesture Module
• Google Search, Google Maps
• File Navigation, Copy/Paste
• System Time, Sleep/Wake, Exit

⚙️ Getting Started

Python Version: 3.8.5

git clone https://github.com/yourusername/gesture-virtual-mouse.git
cd gesture-virtual-mouse
pip install -r requirements.txt
python main.py

📂 Code Structure

gesture-virtual-mouse/
├── main.py
├── gesture_controller/
│   ├── hand_tracker.py
│   ├── glove_detector.py
│   ├── gesture_classifier.py
│   └── mouse_controller.py
├── voice_assistant/
│   ├── proton.py
│   ├── speech_recognizer.py
│   └── command_executor.py
├── utils/
│   ├── config.py
│   └── helpers.py
├── resources/
│   └── models/
├── requirements.txt
└── README.md

📜 Flowchart / 📌 Architecture

Flow: Start → Hand Detection → Landmark Extraction → Gesture Classifier → Mouse/Voice Action

Architecture:

+ Webcam Input
    ↓
+ MediaPipe / Glove Detection
    ↓
+ Landmark / Color Extraction
    ↓
+ Gesture Classifier (CNN / Rules)
    ↓
+ PyAutoGUI (Mouse Actions)
    ↘
+ Proton (Voice Assistant) → System / Web Commands

🔍 Comparative Analysis

Feature	Traditional Mouse	Touch Screen	Gesture Mouse
Hardware	High	Medium	Low
Contactless	No	No	Yes
Voice Control	No	No	Yes
Accessibility	Low	Medium	High
Cost	Medium	Low	High

💻 Language & Tool Roles

• Python - Core development
• MediaPipe - Hand tracking
• OpenCV - Video feed & processing
• PyAutoGUI - Mouse control
• SpeechRecognition - Voice input
• pyttsx3 - Voice output

🔧 System Layers

• Input Layer: Webcam & Microphone
• Processing Layer: ML models & command logic
• Output Layer: Mouse Actions & Voice Response

🔮 Upcoming Features

• Cross-platform support (Linux, Windows)
• Dynamic gesture recognition (LSTM)
• Depth sensor support (e.g., RealSense)
• Multilingual Voice Assistant (Hindi, German, etc.)
• GUI dashboard for config/logs

🧠 Strategic Recommendations

• Deploy with PyInstaller or Electron for full desktop use
• Use ONNX or TensorRT for faster inference
• Great for use in accessibility tech, smart homes, or touchless UIs

© 2025 Gesture Controlled Virtual Mouse. All rights reserved.