Webpage, arXiv

• key_insertion.py: the extremum seeking algorithm
• key_insertion_cma.py: the CMA-ES baseline
• vision_accuracy.py/sh: scripts for computing lock pose from vision and plotting the accuracy
• vision_insertion.sh: conducts a complete vision initialized insertion from start to finish
• realsense.py: hardware interfacing with the Realsense D435i camera used for vision based initialization
• foundationpose/: scripts for running the FoundationPose docker container that was used estimate socket pose

The recorded data for all 360 trials of our method is available here.

Data collected during vision based insertions is available here.

Data collected using the CMA-ES method is available here.

The AutoMate objects with recreated chamfers are available here. Note that object 00015's plug was chamfered correctly in AutoMate's released files and so is not included here.

The 3D meshes used with FoundationPose are availabled here.

• vme_research: A minimal version of an internal library used by the group for robotics research. To install run: pip install -e ./vme_research from the root of this repository.
• numpy, scipy, matplotlib, jax, open3d
• OpenCV
• ur_rtde
• SAM2
• FoundationPose
• pycma