Geometry-Informed Neural Network (GIN) Target Localization

This repository contains the code to detect circular target for "Target Localization and Manipulator Design for Safe Aerial Physical Inspection Mission".

Aerial manipulators, which combine UAVs with robotic arms, are increasingly useful for tasks requiring surface detection and physical interaction. This paper introduces a novel system design for safe Aerial Physical Interaction (APhI) with defined surfaces. The system includes a UAV platform equipped with a 3-DoF lightweight robotic arm, an RGB-D camera for target detection, and sensors for precise localization. A Convolutional Neural Network (CNN) is used to detect targets, and the dataset is enhanced to improve accuracy. Outdoor experiments were conducted in various conditions, focusing on industrial settings to ensure reliable target detection and safe interactions.

Structure of the repo

• Configuration files of the neural network training and its architecture.
• Container files
• Launch files
• ROS nodes
• External libraries
• Weights of the neurons of the network
• Visualize data and images
• Docker files
• Kubernetes configuration files

Setup

Installation

The package has been developed in Ubuntu 20.04 with ROS Noetic. Support for other distro is not guarantee.

1. Install ROS Noetic
2. Create a catking workspace

mkdir -p ~/catkin_ws/src
cd ~/catkin_ws
catkin init

3. Clone GIN

cd ~/catkin_ws
git clone git@github.com:Flying-Touchman/gin_target_localization.git

4. Install GIN dependencies (rosdep)

cd ~/catkin_ws
rosdep install --from-paths src --ignore-src -r -y
cd ~/catkin_ws/src/gin_target_localization/docker
pip3 install -r requirements.txt 

5. Build GIN

catkin build -j4 gin

Docker

To instead use docker

1. Install docker, nvidia-docker and docker compose

2. Go inside docker folder

cd gin_target_localization/docker

3. Build docker image

docker compose build gin

4. Create docker container and launch it

docker compose up -d gin

5. Create a terminal inside docker container

docker exec -it gin_cnt zsh

Launch GIN

Launch GIN

roslaunch gin target_detection.launch

Sensor disclaimer

The GIN algorithm has been tested and validated with Realsense Depth Camera D455.
However, the algorithm is designed to process data from different depth camera that publishes the following ROS topics:

• camera info topic
• camera color topic
• camera depth topic

Run GIN with Dataset

Launch GIN with rosbag. Sample rosbag can be download here

roslaunch gin target_detection_rosbag.launch rosbag_path:=<example_bag_path.bag>