🚁 Dronoid - Drones that Understand Natural Language

Vision: Open-source drone operating system with natural language control and computer vision for autonomous facade cleaning.

🎯 Project Goals

Universal Drone OS - Hardware-agnostic system that auto-detects sensors/actuators
Natural Language Control - Fly drones with plain text commands ("takeoff to 10m", "land")
Facade Cleaning - Autonomous dirt detection and cleaning with mounted water system
Simulation First - Test everything in PX4 SITL + Gazebo before hardware deployment
Budget-Friendly - Start with cheap components ($50-200), scalable to industrial ($$$)

✨ Features

🗣️ Natural Language Interface: Control drone with text commands (English/German)
- "takeoff" / "abheben"
- "fly up to 10m" / "fliege hoch auf 10m"
- "land" / "landen"
🤖 PX4 SITL Integration: Full simulation environment with Gazebo
🔌 ROS2 Bridge: DDS communication for advanced features
🎯 Vision-Ready: Prepared for VLM integration (Wall-X, TinyML)
📦 Plug & Play: YAML-based hardware configuration

⚡ TL;DR - Fastest Start

git clone https://github.com/enving/Dronoid.git
cd Dronoid
./setup_dependencies.sh  # Download PX4, build deps (~5 min)
./px4-client             # Auto-starts everything!

Type takeoff and watch your drone fly! 🚁

🚀 Quick Start (5 Minutes)

1. Install Dependencies

# Install ROS2 Jazzy (if not already installed)
sudo apt update
sudo apt install -y ros-jazzy-desktop

# Install PX4 dependencies
sudo apt install -y \
    python3-pip \
    git \
    wget \
    gnutls-bin \
    openjdk-11-jre-headless \
    gz-harmonic

# Install Python packages
pip3 install --break-system-packages \
    pymavlink \
    python-dotenv \
    requests \
    kconfiglib \
    pyros-genmsg \
    packaging \
    toml \
    numpy \
    jinja2

# Or use requirements.txt:
pip3 install --break-system-packages -r requirements.txt

2. Clone and Setup

git clone https://github.com/YOUR_USERNAME/Dronoid.git
cd Dronoid

# Download dependencies (PX4, DDS Agent, etc.)
chmod +x setup_dependencies.sh
./setup_dependencies.sh

3. Start the Drone System

./start_drone.sh

This will:

Start PX4 SITL (Software In The Loop)
Launch Gazebo Harmonic simulator
Spawn X500 quadcopter drone
Wait for "Ready for takeoff!" message

4. Control the Drone

Easy Mode (auto-starts PX4 if needed):

./px4-client

Manual Mode:

# Make sure PX4 is running first
./start_drone.sh

# Then start control
python3 nl_drone.py

Then type natural language commands:

takeoff - Take off to 5m altitude
takeoff to 10m - Take off to 10m
fly up / abheben - Take off (German)
land / landen - Land the drone
q - Quit

🏗️ Architecture

┌─────────────────────────────────────┐
│  Natural Language Interface          │
│  (nl_drone.py)                      │
│  • Text parsing (Regex/Cloud LLM)   │
│  • Bilingual support (EN/DE)        │
└──────────────┬──────────────────────┘
               │ MAVLink UDP
               ▼
┌─────────────────────────────────────┐
│  PX4 Autopilot (SITL)               │
│  • Flight controller firmware       │
│  • MAVLink communication            │
│  • UDP Port 14580 (Offboard API)    │
└──────────────┬──────────────────────┘
               │ Gazebo Bridge
               ▼
┌─────────────────────────────────────┐
│  Gazebo Harmonic Simulator          │
│  • Physics simulation               │
│  • X500 quadcopter model            │
│  • 3D visualization                 │
└─────────────────────────────────────┘

┌─────────────────────────────────────┐
│  ROS2 Integration (Future)          │
│  • Micro-XRCE-DDS-Agent             │
│  • px4_msgs                         │
│  • dronecore package                │
└─────────────────────────────────────┘

📁 Project Structure

Humanoid/
├── README.md                       # This file
├── start_drone.sh                  # Quick start script
├── nl_drone.py                     # Natural language control
├── control_drone.py                # Numeric menu control
├── quick_test.sh                   # PX4 + Gazebo launcher
│
├── PX4-Autopilot/                  # PX4 firmware (v1.15.4)
│   ├── Tools/simulation/gz/        # Gazebo models & worlds
│   └── build/px4_sitl_default/     # Build artifacts (git ignored)
│
├── Micro-XRCE-DDS-Agent/           # ROS2 ↔ PX4 bridge
│   └── build/                      # Build artifacts (git ignored)
│
├── px4_msgs/                       # ROS2 message definitions
├── dronecore/                      # ROS2 control package
│   ├── drone_commander.py          # ROS2 node for commands
│   └── nl_bridge.py                # FastAPI HTTP server
│
└── old_docs/                       # Legacy documentation

🎮 Control Methods

1. Natural Language (Recommended)

python3 nl_drone.py

Text commands with intelligent parsing.

2. Numeric Menu

python3 control_drone.py

Simple 1-4 menu for basic commands.

3. ROS2 Commands (Advanced)

# Build dronecore package first
cd dronecore && colcon build
source install/setup.bash

# Launch ROS2 nodes
ros2 run dronecore drone_commander

🧠 Natural Language Processing

Dronoid supports two modes for understanding your commands:

1. Regex Parser (Default - Lightweight)

⚡ Fast: Instant response
💻 No internet needed: Works offline
🪶 Lightweight: No ML model, runs on any hardware
📝 Pattern matching: "takeoff" → takeoff()

Perfect for: Weak laptops, offline use, predictable commands

2. Cloud LLM via OpenRouter (Optional - Smart)

🤖 Intelligent: Understands context and synonyms
🌍 Multilingual: Works in any language
💡 Flexible: "fly a bit higher" → understands intent
🆓 Free tier available: Google Gemini Flash 1.5

Setup (optional):

# 1. Get free API key: https://openrouter.ai/keys
# 2. Copy example config
cp .env.example .env

# 3. Edit .env and add your key
nano .env  # or use any text editor

The system automatically falls back to regex if OpenRouter is unavailable!

🛡️ Robust Connection Handling

Dronoid includes smart connection management to make your experience smooth:

Auto-Recovery Features

10 Retry Attempts: Automatically retries connection if PX4 is starting
Intelligent Port Detection: Tests multiple MAVLink ports
Clear Error Messages: Helpful troubleshooting hints if connection fails
Auto-Start Wrapper: px4-client checks if PX4 is running and starts it automatically

How It Works

./px4-client
  ↓
Checks: Is PX4 running?
  ├─ Yes → Connect to drone ✓
  └─ No  → Start PX4 → Wait for ready → Connect ✓

This means you never have to manually start PX4 - just run ./px4-client and fly!

🔧 Configuration

Environment Variables

Create .env file (not tracked in git):

# Optional: Cloud LLM for advanced NL processing
OPENROUTER_API_KEY=your-key-here

Hardware Configuration (Future)

YAML-based plug & play:

# drone_config.yaml
hardware:
  flight_controller: px4
  camera:
    type: raspberry_pi_camera_v2
    resolution: 1920x1080
  water_system:
    pump: 12v_dc
    pressure_sensor: analog

📊 System Requirements

Minimum (Simulation)

CPU: Dual-core 2.0 GHz
RAM: 4GB (note: Gazebo can be heavy)
Storage: 10GB free
OS: Ubuntu 24.04 (or 22.04)

Recommended (Hardware + Vision)

CPU: Quad-core 2.5 GHz+
RAM: 8GB+
GPU: For VLM inference (optional)
Additional: Raspberry Pi 4 (4GB) for onboard processing

🎯 Roadmap

✅ Phase 1: COMPLETED

PX4 SITL + Gazebo setup
MAVLink communication
Natural language control (regex-based)
Basic commands (arm, takeoff, land)

🚧 Phase 2: IN PROGRESS

Cloud LLM integration (OpenRouter)
Advanced command parsing
HTTP API for remote control
ROS2 full integration

📋 Phase 3: PLANNED

VLM integration (Wall-X / TinyML)
Dirt detection on facades
Water system control
Autonomous cleaning routines

📋 Phase 4: FUTURE

Hardware deployment (Raspberry Pi + PX4)
Real-world testing
Multi-drone coordination
Industrial scalability

🐛 Troubleshooting

Drone won't connect

# Check if PX4 is running
ps aux | grep px4

# Restart system
./start_drone.sh

Gazebo GUI not showing drone

# Check environment variables
echo $GZ_SIM_RESOURCE_PATH

# Should include PX4 models path
export GZ_SIM_RESOURCE_PATH=$PWD/PX4-Autopilot/Tools/simulation/gz/models:$PWD/PX4-Autopilot/Tools/simulation/gz/worlds

Python connection timeout

# PX4 MAVLink should be on port 14580
lsof -i UDP:14580

# If port is closed, PX4 isn't running

🤝 Contributing

This is an open-source project! Contributions welcome:

🐛 Bug reports via GitHub Issues
💡 Feature suggestions
🔧 Pull requests
📖 Documentation improvements

📚 Related Projects

PX4 Autopilot - Drone flight controller
MAVSDK - Drone API library
Wall-X - VLM for visual inspection
Dronekit-Python - Similar project

📝 License

MIT License - See LICENSE file

👤 Author

Tristan - Universal Drone OS Project

Built with Claude Code 🤖

Note: This project is in active development. Star ⭐ the repo to follow progress!

Name		Name	Last commit message	Last commit date
Latest commit History 4 Commits
dronecore		dronecore
.env.example		.env.example
.gitignore		.gitignore
ARCHITECTURE_v2.md		ARCHITECTURE_v2.md
DRONE_PROJECT_PLAN.md		DRONE_PROJECT_PLAN.md
DRONE_README.md		DRONE_README.md
DRONE_SETUP_GUIDE.md		DRONE_SETUP_GUIDE.md
DRONE_SETUP_STATUS.md		DRONE_SETUP_STATUS.md
IMPLEMENTATION_EXAMPLE.md		IMPLEMENTATION_EXAMPLE.md
OPENROUTER_SETUP.md		OPENROUTER_SETUP.md
PROJECT_STATUS.md		PROJECT_STATUS.md
README.md		README.md
control_drone.py		control_drone.py
nl_drone.py		nl_drone.py
px4-client		px4-client
quick_test.sh		quick_test.sh
requirements.txt		requirements.txt
setup_dependencies.sh		setup_dependencies.sh
start_drone.sh		start_drone.sh

enving/Dronoid

Folders and files

Latest commit

History

Repository files navigation