Skip to content

quickstart.md

Latest commit

 

History

History
66 lines (44 loc) · 3.02 KB

quickstart.md

File metadata and controls

66 lines (44 loc) · 3.02 KB

Quickstart Guide

This guide will help you get the Mobile Manipulator Demo up and running quickly using Docker Compose.

Important

This guide is designed for running the demo with cloud-based models for a smooth and easy setup. If you’d like to use local models instead, please refer to the Setup Guide. Note: Running local models requires approximately 48GB of VRAM.

Prerequisites

1. Prepare the environment

1.1 Clone the repository

git clone https://github.com/RobotecAI/agentic-mobile-manipulator.git
cd agentic-mobile-manipulator

1.2 Configure the LLM/VLM

Tip

The repository contains two configuration files:

  • config.toml: Configuration for the LLM/VLM models.
  • cloud_config.toml: Configuration for the cloud LLM/VLM models.

The cloud_config.toml file is used to configure the LLM/VLM models. By default, the demo uses OpenAI. If you want to use other models, modify the config.toml and compose.yaml (vendor api key environment variable) accordingly.

2. Run the demo

2.1 Allow local connections to your X server

Before launching the containers, you need to allow local connections to your X server so the GUI applications (O3DE simulation and HMI) can display on your screen.

Run the following command in your terminal:

xhost +local:docker

2.2 Run the demo

docker compose -f docker/compose.yaml up

What to Expect

Demo Windows

  1. Simulation Window: The O3DE simulation window will appear, showing the warehouse environment. Use "Control -> Simulation Scenarios" to spawn objects in the warehouse. Use "Control -> On demand predefined tasks" to run predefined tasks. Alternatively, publish your own tasks to the /user_tasks topic.
  2. HMI: A separate window or terminal output indicating the HMI is running will appear. Spawn
  3. Agents: The autonomous agents will start in the background.

Troubleshooting

  • O3DE Doesn't start: Please make sure your GPU drivers and container toolkit are properly installed. For more information about what might be going wrong, try running docker compose -f docker/compose.yaml up sim to check the logs. If the issue persists, perform a graceful shutdown of the container/docker compose and then restart it.
  • Display Issues: If windows don't appear, ensure you ran xhost +local:docker and that your DISPLAY environment variable is set correctly (echo $DISPLAY).
  • GPU Access: If the simulation runs slowly or crashes, check your GPU drivers and container toolkit installation.
  • AMD GPU Access: Ensure you have passed the correct devices. For more details on running ROCm Docker containers, see the official AMD documentation.