Tello Vision v2.0 🚁🤖

Modern, modular instance segmentation and object detection for DJI Tello drones. Complete rewrite with SOTA models, clean architecture, and actual performance.

📖 Documentation Index

Section	Description
Quickstart	Get up and running fast
Migration	Notes on upgrading between versions
Improvements	Roadmap and ideas for future releases
License	License and legal details

Features

• 🎯 Multiple Detection Backends: YOLOv8 (fast) or Detectron2 (accurate)
• 🚁 Modern Drone Control: Smooth RC controls, async processing
• 🎨 Rich Visualization: Masks, bboxes, labels with transparency
• 📹 Recording & Photos: Video recording and frame capture
• ⚙️ Config-Driven: YAML configuration for everything
• 🔌 Pluggable Architecture: Easy to add custom models
• 📊 Real-time Stats: FPS, battery, detection count

Quick Start

1. Installation

# Clone the repo
git clone <your-repo-url>
cd tello_vision

# Create virtual environment
python3.10 -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

# Install with YOLOv8 (recommended for real-time)
pip install -e ".[yolo]"

# OR install with Detectron2 (higher quality, slower)
pip install -e ".[detectron2]"

# OR both
pip install -e ".[yolo,detectron2]"

2. Test Detection Without Drone

Good for verifying everything works:

python examples/test_detector.py --source 0  # Webcam

3. Benchmark Your Setup

See what FPS you can get:

python examples/benchmark.py

4. Configure

Edit config.yaml:

detector:
  backend: "yolov8" # or "detectron2"
  yolov8:
    model: "yolov8n-seg.pt" # nano = fastest
    confidence: 0.5
    device: "cuda" # or "cpu"

5. Test Detection With Drone

# Make sure your Tello is powered on and connected to its WiFi
python -m tello_vision.app

# With custom config
python -m tello_vision.app --config my_config.yaml

Controls

Action	Key
Takeoff	Tab
Land	Backspace
Emergency Stop	Esc
Move	W/A/S/D
Up/Down	Space/Shift
Rotate	Q/E
Record Video	R
Take Photo	Enter
Quit	P

Architecture

tello_vision/
├── detectors/
│   ├── base_detector.py      # Abstract detector interface
│   ├── yolo_detector.py       # YOLOv8 implementation
│   └── detectron2_detector.py # Detectron2 implementation
├── tello_controller.py        # Drone control & video streaming
├── visualizer.py              # Detection visualization
└── app.py                     # Main application

Adding Custom Models

Extend BaseDetector:

from tello_vision.detectors.base_detector import BaseDetector, DetectionResult

class MyCustomDetector(BaseDetector):
    def load_model(self):
        # Load your model
        pass

    def detect(self, frame) -> DetectionResult:
        # Run inference
        pass

Register in base_detector.py:

def create_detector(backend: str, config: dict):
    if backend == 'custom':
        from .my_custom_detector import MyCustomDetector
        return MyCustomDetector(config)

Performance Comparison

Model	Device	FPS	mAP	Use Case
YOLOv8n-seg	RTX 3060	25-30	~35	Real-time, fast
YOLOv8s-seg	RTX 3060	18-22	~38	Balanced
YOLOv8m-seg	RTX 3060	12-15	~41	Accuracy focus
Detectron2 R50	RTX 3060	8-12	~38	High quality
YOLOv8n-seg	CPU	2-3	~35	CPU fallback

FPS measured at 960x720 resolution

Configuration Guide

Target Specific Objects

detector:
  target_classes: ["person", "car", "dog"] # Only detect these

Adjust Visualization

visualization:
  show_masks: true
  mask_alpha: 0.4 # Transparency
  show_boxes: true
  box_thickness: 2
  show_confidence: true

Performance Tuning

processing:
  frame_skip: 1 # Process every 2nd frame (doubles FPS)
  async_inference: true # Run detection in separate thread
  max_queue_size: 3

Recording

processing:
  record_video: false # Auto-start recording
  output_dir: "./output"
  video_codec: "mp4v"

Advanced Usage

Python API

from tello_vision import TelloVisionApp

app = TelloVisionApp('config.yaml')
if app.initialize():
    app.run()

Custom Processing Pipeline

from tello_vision import TelloController, BaseDetector, Visualizer

# Initialize components
drone = TelloController(config)
detector = BaseDetector.create_detector('yolov8', config)
visualizer = Visualizer(config)

# Custom loop
drone.connect()
detector.load_model()

while True:
    frame = drone.get_frame()
    result = detector.detect(frame)

    # Custom logic here
    for det in result.detections:
        if det.class_name == 'person' and det.confidence > 0.8:
            print(f"Person detected at {det.center}")

    frame = visualizer.draw_detections(frame, result)

Self-Driving Car Extensions

Since you're exploring autonomous vehicles, here are some ideas:

1. Object Tracking

Add a tracker to follow specific objects:

# Use ByteTrack or SORT
from boxmot import ByteTrack

tracker = ByteTrack()
tracks = tracker.update(detections, frame)

2. Path Planning

Integrate with planning algorithms:

if 'person' in detected_classes:
    drone.move_left(30)  # Avoid obstacle

3. SLAM Integration

Connect with ORB-SLAM or similar:

from orbslam3 import System

slam = System('vocab.txt', 'tello.yaml')
pose = slam.process_image_mono(frame, timestamp)

4. Semantic Segmentation

Add depth estimation or semantic maps for better navigation.

Troubleshooting

Connection Issues

# Check WiFi connection
ping 192.168.10.1

# Verify Tello firmware is up to date
# Check battery > 10%

Performance Issues

• Use smaller model: yolov8n-seg.pt instead of yolov8x-seg.pt
• Enable frame skipping: frame_skip: 1 or 2
• Lower confidence threshold may increase speed
• Use GPU if available

Import Errors

# YOLOv8
pip install ultralytics

# Detectron2 (Linux/Mac)
pip install 'git+https://github.com/facebookresearch/detectron2.git'

# Detectron2 (Windows) - build from source or use pre-built wheels

Development

# Install dev dependencies
pip install -e ".[dev]"

# Format code
black tello_vision/

# Lint
ruff check tello_vision/

# Type check
mypy tello_vision/

Roadmap

• Multi-drone support
• ROS2 integration
• Web interface
• Object tracking (ByteTrack)
• Autonomous navigation
• Dataset recording tool
• Model training pipeline
• Docker container

License

Apache License, Version 2.0 - fork it, break it, make it better.

Acknowledgments

• Original repo: dronefreak/dji-tello-object-detection-segmentation
• Ultralytics YOLOv8
• Detectron2
• djitellopy

---