Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 14 additions & 14 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# AgileX Piper Robot Teleoperation with NeuraCore
# AgileX Piper Robot Teleoperation with Neuracore

This project is a complete example showcasing how to use Neuracore with the AgileX Piper robot. The project provides examples teleoperating the AgileX Piper robot using a Meta Quest controller, collecting demonstration data with [Neuracore](https://neuracore.com/), deploying trained policies, and an easy interface to tune most of the associated parameters.

Expand Down Expand Up @@ -88,13 +88,13 @@ python examples/1_tune_teleop_params.py [--ip-address <quest-ip>]

**Script**: `examples/2_collect_teleop_data_with_neuracore.py`

Now that you have a well-tuned setup, you can use this script to record teleoperation demonstrations to NeuraCore.
Now that you have a well-tuned setup, you can use this script to record teleoperation demonstrations to Neuracore.

```bash
python examples/2_collect_teleop_data_with_neuracore.py [--ip-address <quest-ip>] [--dataset-name <name>]
```

**Note**: You must be logged into NeuraCore.
**Note**: You must be logged into Neuracore.

**Arguments**:
- `--ip-address`: IP address of Meta Quest device (optional). Only needed when using WiFi connection. If not provided, defaults to auto-discovery via USB.
Expand All @@ -104,25 +104,25 @@ python examples/2_collect_teleop_data_with_neuracore.py [--ip-address <quest-ip>
- Same as script 1, plus:
- **Right Joystick Press**: Start/stop data recording

### 3. Replay NeuraCore Episodes
### 3. Replay Neuracore Episodes

**Script**: `examples/3_replay_neuracore_episodes.py`

Replay recorded episodes from a NeuraCore dataset on the physical robot.
Replay recorded episodes from a Neuracore dataset on the physical robot.

```bash
python examples/3_replay_neuracore_episodes.py --dataset-name <dataset-name> [--frequency <hz>] [--episode-index <index>]
```

**Arguments**:
- `--dataset-name`: Name of the NeuraCore dataset to replay
- `--dataset-name`: Name of the Neuracore dataset to replay
- `--frequency`: Playback frequency in Hz (default: 0). 0 plays the data aperiodically (not synchronized at a certain frequency as it was recorded).
- `--episode-index`: Which episode to replay (default: 0). -1 will start replaying all the episodes one after the other.

**NOTE:** please be careful that the robot **will start moving** on the same trajectory that was recorded. Pressing `ctrl+C`
will gracefully disable the robot and it will cut power to the motors after 5 seconds.

### 4. Rollout NeuraCore Policy (Full GUI)
### 4. Rollout Neuracore Policy (Full GUI)

**Script**: `examples/4_rollout_neuracore_policy.py`

Expand All @@ -147,12 +147,12 @@ python examples/4_rollout_neuracore_policy.py --model-path <path-to-model> [--ip
```

**Arguments**:
- `--train-run-name`: Name of the NeuraCore training run (fetches model from NeuraCore)
- `--train-run-name`: Name of the Neuracore training run (fetches model from Neuracore)
- `--model-path`: Local path to model file (alternative to train-run-name)
- `--ip-address`: IP address of Meta Quest device (optional). Only needed when using WiFi connection. If not provided, defaults to auto-discovery via USB.


### 5. Rollout NeuraCore Policy (Minimal)
### 5. Rollout Neuracore Policy (Minimal)

**Script**: `examples/5_rollout_neuracore_policy_minimal.py`

Expand All @@ -169,7 +169,7 @@ python examples/5_rollout_neuracore_policy_minimal.py --model-path <path-to-mode
```

**Arguments**:
- `--train-run-name`: Name of the NeuraCore training run (fetches model from NeuraCore)
- `--train-run-name`: Name of the Neuracore training run (fetches model from Neuracore)
- `--model-path`: Local path to model file (alternative to train-run-name)

### 6. Visualize Policy from Dataset
Expand All @@ -189,8 +189,8 @@ python examples/6_visualize_policy_from_dataset.py --dataset-name <dataset-name>
```

**Arguments**:
- `--dataset-name`: Name of the NeuraCore dataset to visualize
- `--train-run-name`: Name of the NeuraCore training run (fetches model from NeuraCore)
- `--dataset-name`: Name of the Neuracore dataset to visualize
- `--train-run-name`: Name of the Neuracore training run (fetches model from Neuracore)
- `--model-path`: Local path to model file (alternative to train-run-name)

## Configuration
Expand Down Expand Up @@ -248,10 +248,10 @@ example_agilex/
- Check robot power and CAN bus connection
- Ensure robot is in the correct mode for control

### NeuraCore Connection Issues
### Neuracore Connection Issues

- Verify you're logged in: `neuracore login`
- Check network connectivity to NeuraCore servers
- Check network connectivity to Neuracore servers
- Verify dataset/run names are correct

## Safety Notes
Expand Down
22 changes: 11 additions & 11 deletions examples/2_collect_teleop_data_with_neuracore.py
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
#!/usr/bin/env python3
"""Piper Robot Teleoperation with Meta Quest Controller and NeuraCore data collection.
"""Piper Robot Teleoperation with Meta Quest Controller and Neuracore data collection.

This demo uses Pink IK control with Meta Quest controller input to control the Piper robot and
logs data to NeuraCore.
logs data to Neuracore.
"""


Expand Down Expand Up @@ -64,13 +64,13 @@


def neuracore_logging_worker(queue: Queue, worker_id: int) -> None:
"""Worker process that logs data to NeuraCore from the queue.
"""Worker process that logs data to Neuracore from the queue.

Args:
queue: Multiprocessing queue containing logging tasks
worker_id: Worker process identifier
"""
# Initialize NeuraCore connection in this worker process
# Initialize Neuracore connection in this worker process
try:
nc.login()
nc.connect_robot(
Expand All @@ -80,7 +80,7 @@ def neuracore_logging_worker(queue: Queue, worker_id: int) -> None:
overwrite=False,
)
except Exception as e:
print(f"\n⚠️ Worker {worker_id} failed to login to NeuraCore: {e}")
print(f"\n⚠️ Worker {worker_id} failed to login to Neuracore: {e}")
return

print(f"\n👷 Logging worker process {worker_id} started")
Expand All @@ -101,7 +101,7 @@ def neuracore_logging_worker(queue: Queue, worker_id: int) -> None:
# Unpack task: (function_name, args_tuple, timestamp)
function_name, data_value, timestamp = task

# Call appropriate NeuraCore logging function
# Call appropriate Neuracore logging function
try:
if function_name == "log_joint_positions":
data_value = np.radians(data_value)
Expand Down Expand Up @@ -132,7 +132,7 @@ def neuracore_logging_worker(queue: Queue, worker_id: int) -> None:
else:
print(f"\n⚠️ Unknown logging function: {function_name}")
except Exception as e:
print(f"\n⚠️ Failed to log {function_name} to NeuraCore: {e}")
print(f"\n⚠️ Failed to log {function_name} to Neuracore: {e}")

# Sleep to maintain loop rate
elapsed = time.time() - start_time
Expand Down Expand Up @@ -247,7 +247,7 @@ def on_button_rj_pressed() -> None:
multiprocessing.set_start_method("spawn")

parser = argparse.ArgumentParser(
description="Piper Robot Teleoperation with NeuraCore Data Collection - REAL ROBOT CONTROL"
description="Piper Robot Teleoperation with Neuracore Data Collection - REAL ROBOT CONTROL"
)
parser.add_argument(
"--ip-address",
Expand All @@ -273,8 +273,8 @@ def on_button_rj_pressed() -> None:
print(f" 📸 Camera Frame: {CAMERA_FRAME_STREAMING_RATE} Hz")
print(f" 📊 Joint State: {JOINT_STATE_STREAMING_RATE} Hz")

# Connect to NeuraCore
print("\n🔧 Initializing NeuraCore...")
# Connect to Neuracore
print("\n🔧 Initializing Neuracore...")
nc.login()
nc.connect_robot(
robot_name="AgileX PiPER",
Expand All @@ -293,7 +293,7 @@ def on_button_rj_pressed() -> None:
)

# Initialize logging queue and worker pool
print("\n📝 Initializing NeuraCore logging queue and worker pool...")
print("\n📝 Initializing Neuracore logging queue and worker pool...")
logging_queue: Queue = Queue()
logging_workers = []

Expand Down
10 changes: 5 additions & 5 deletions examples/4_rollout_neuracore_policy.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
#!/usr/bin/env python3
"""Piper Robot Test with NeuraCore policy.
"""Piper Robot Test with Neuracore policy.

This script loads a trained NeuraCore policy, reads status from the piper robot
This script loads a trained Neuracore policy, reads status from the piper robot
controlled by the Meta Quest controller, and replays the prediction horizon virtually
on Viser to test the stability of the policy output.
"""
Expand Down Expand Up @@ -656,7 +656,7 @@ def update_visualization(

if __name__ == "__main__":
parser = argparse.ArgumentParser(
description="Piper Robot Test with NeuraCore Policy - REAL ROBOT CONTROL"
description="Piper Robot Test with Neuracore Policy - REAL ROBOT CONTROL"
)
parser.add_argument(
"--ip-address",
Expand Down Expand Up @@ -695,8 +695,8 @@ def update_visualization(
print(f" 📊 Joint State: {JOINT_STATE_STREAMING_RATE} Hz")
print(f" 🖥️ Visualization: {VISUALIZATION_RATE} Hz")

# Connect to NeuraCore
print("\n🔧 Initializing NeuraCore...")
# Connect to Neuracore
print("\n🔧 Initializing Neuracore...")
nc.login()
nc.connect_robot(
robot_name="AgileX PiPER",
Expand Down
10 changes: 5 additions & 5 deletions examples/5_rollout_neuracore_policy_minimal.py
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,7 @@ def convert_predictions_to_horizon_dict(predictions: dict) -> dict[str, list[flo


def log_current_state(data_manager: DataManager) -> None:
"""Log current state to NeuraCore."""
"""Log current state to Neuracore."""
current_joint_angles = data_manager.get_current_joint_angles()
if current_joint_angles is None:
print("⚠️ No joint angles available")
Expand All @@ -91,13 +91,13 @@ def log_current_state(data_manager: DataManager) -> None:
print("⚠️ No RGB image available")
return

# Prepare data for NeuraCore logging
# Prepare data for Neuracore logging
joint_angles_rad = np.radians(current_joint_angles)
joint_positions_dict = {
joint_name: angle for joint_name, angle in zip(JOINT_NAMES, joint_angles_rad)
}

# Log joint positions, parallel gripper open amounts, and RGB image to NeuraCore
# Log joint positions, parallel gripper open amounts, and RGB image to Neuracore
nc.log_joint_positions(joint_positions_dict)
nc.log_parallel_gripper_open_amount(GRIPPER_LOGGING_NAME, gripper_open_value)
nc.log_rgb(CAMERA_LOGGING_NAME, rgb_image)
Expand Down Expand Up @@ -214,8 +214,8 @@ def execute_horizon(
print("PIPER POLICY ROLLOUT")
print("=" * 60)

# Initialize NeuraCore
print("\n🔧 Initializing NeuraCore...")
# Initialize Neuracore
print("\n🔧 Initializing Neuracore...")
nc.login()
nc.connect_robot(
robot_name="AgileX PiPER",
Expand Down
12 changes: 6 additions & 6 deletions examples/6_visualize_policy_from_dataset.py
Original file line number Diff line number Diff line change
Expand Up @@ -52,8 +52,8 @@
if (args.train_run_name is None) == (args.model_path is None):
parser.error("Exactly one of --train-run-name or --model-path must be provided")

# Connect to NeuraCore
print("🔧 Initializing NeuraCore...")
# Connect to Neuracore
print("🔧 Initializing Neuracore...")
nc.login()
nc.connect_robot(robot_name="AgileX PiPER", urdf_path=str(URDF_PATH), overwrite=False)

Expand Down Expand Up @@ -174,7 +174,7 @@ def select_random_state() -> None:
for joint_name in JOINT_NAMES:
if joint_name in joint_data:
joint_positions_dict[joint_name] = joint_data[joint_name].value
# Log to NeuraCore for visualization
# Log to Neuracore for visualization
nc.log_joint_positions(joint_positions_dict)

# Extract gripper
Expand All @@ -183,7 +183,7 @@ def select_random_state() -> None:
gripper_data = step.data[DataType.PARALLEL_GRIPPER_OPEN_AMOUNTS]
if GRIPPER_LOGGING_NAME in gripper_data:
gripper_value = gripper_data[GRIPPER_LOGGING_NAME].open_amount
# Log to NeuraCore for visualization
# Log to Neuracore for visualization
nc.log_parallel_gripper_open_amount(GRIPPER_LOGGING_NAME, gripper_value)

# Extract RGB image
Expand All @@ -196,7 +196,7 @@ def select_random_state() -> None:
image_pil = Image.fromarray(rgb_image)
image_pil.save("current_image.png")
print("💾 Saved image to current_image.png")
# Log to NeuraCore for visualization
# Log to Neuracore for visualization
nc.log_rgb(CAMERA_LOGGING_NAME, rgb_image)

# Get policy prediction
Expand Down Expand Up @@ -264,7 +264,7 @@ def select_random_state() -> None:
)
urdf_vis.update_cfg(joint_config)

# Log to NeuraCore for visualization
# Log to Neuracore for visualization
# NOTE: we log to joint positions instead of joint target positions
# because the latter is not visualized by Neuracore
joint_config_dict = {
Expand Down
2 changes: 1 addition & 1 deletion examples/common/configs.py
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@
VISUALIZATION_RATE = 60.0 # GUI updates
ROBOT_RATE = 100.0

# NeuraCore data collection rates
# Neuracore data collection rates
JOINT_STATE_STREAMING_RATE = 100.0 # Data collection rate for neuracore
CAMERA_FRAME_STREAMING_RATE = 60.0 # Data collection rate for camera frame

Expand Down