LEAD: Minimizing Learner–Expert Asymmetry in End-to-End Driving

Project Page · Documentation · Weights · Paper · Supplementary Material

Official implementation of LEAD and TransFuser v6, an expert-student policy pair for autonomous driving research in CARLA. Includes a complete pipeline for data collection, training, and closed-loop evaluation.

Main Features

LEAD provides a comprehensive framework for end-to-end driving research in the CARLA simulator, featuring TransFuser v6 — a simple yet state-of-the-art model.

Key highlights:

• Data-centric infrastructure:
  • Always know what type and shape your tensors have, enforced with BearType and JaxTyping.
  • Extensive visualizations for debugging data pipelines and closed-loop evaluation.
  • Compact datasets with lower storage overhead (72h of driving fits in ~200GB).
• Scalable experiment management:
  • Built-in SLURM wrapper with unified naming across SLURM jobs, WandB runs, and output directories.
  • Each training/evaluation is a versioned bash script, ensuring full reproducibility.
• Cross-dataset training:
  • Training and evaluation support for NAVSIM and Waymo datasets.
  • Co-training on synthetic CARLA data.

Table of Contents

• Roadmap
• Updates
• Setup Project
• Quick Start
• Bench2Drive Results
• Documentation and Resources
• External Resources
• Acknowledgements
• Citation
• License

Roadmap

• ✅ Checkpoints and inference code (stable)
• 🚧 Documentation, training pipeline and expert code (partial release)
• Full dataset release on HuggingFace
• Cross-dataset training tools and documentation

Status: Active development. Core code and checkpoints are released; remaining components coming soon.

Updates

• 2025/12/24 Arxiv paper and code release

Setup Project

⏱️ 15 minutes

1. Clone project

git clone https://github.com/autonomousvision/lead.git
cd lead

2. Setup environment variables

Set the project root directory and configure paths for CARLA, datasets, and dependencies.

{
  echo
  echo "export LEAD_PROJECT_ROOT=$(pwd)"
  echo "source $(pwd)/scripts/main.sh"
} >> ~/.bashrc

source ~/.bashrc

For Zsh

{
  echo
  echo "export LEAD_PROJECT_ROOT=$(pwd)"
  echo "source $(pwd)/scripts/main.sh"
} >> ~/.zshrc

source ~/.zshrc

3. Create python environment

We use Miniconda for this project

# Install conda-lock
pip install conda-lock

# Create Conda environment
conda-lock install -n lead conda-lock.yml

# Activate conda environment
conda activate lead

Install dependencies with uv

# Install uv
pip install uv

# Install dependencies
uv pip install -r requirements.txt

# Install project
uv pip install -e .

Alternative: vanilla pip

# Install dependencies
pip install -r requirements.txt

# Install project
pip install -e . 

4. Setup CARLA

Install CARLA 0.9.15 at 3rd_party/CARLA_0915

bash scripts/setup_carla.sh

Or softlink existing CARLA

ln -s /your/carla/path $LEAD_PROJECT_ROOT/3rd_party/CARLA_0915

5. Further setup

# Set-up git hooks
pre-commit install

# Install other tools
conda install conda-forge::ffmpeg conda-forge::parallel conda-forge::tree

Note

We also provide a minimal docker compose setup (not extensively tested yet) here.

Quick Start

⏱️ 5 minutes

1. Download model checkpoints

We provide pre-trained checkpoints on HuggingFace for reproducibility.

Checkpoint	Description	Bench2Drive	Longest6 v2	Town13
tfv6_regnety032	TFv6	95.2	62	5.01
tfv6_resnet34	ResNet34 Backbone	94.7	57	3.31
4cameras_resnet34	Additional rear camera	95.1	53	-
noradar_resnet34	No radar sensor	94.7	52	-
visiononly_resnet34	Vision-only driving model	91.6	43	-
town13heldout_resnet34	Generalization evaluation	93.1	52	2.65

To download one checkpoint:

mkdir -p outputs/checkpoints/tfv6_resnet34
wget https://huggingface.co/ln2697/TFv6/resolve/main/tfv6_resnet34/config.json -O outputs/checkpoints/tfv6_resnet34/config.json
wget https://huggingface.co/ln2697/TFv6/resolve/main/tfv6_resnet34/model_0030_0.pth -O outputs/checkpoints/tfv6_resnet34/model_0030_0.pth

Alternatively, to download all checkpoints at once with git lfs:

git clone https://huggingface.co/ln2697/TFv6 outputs/checkpoints
cd outputs/checkpoints
git lfs pull

2. Run model evaluation

See evaluation configuration at config_closed_loop. Turn off the options produce_demo_video and produce_debug_video for faster evaluation. By default, the pipeline loads all three seeds of a checkpoint as an ensemble. If memory is a problem, simply change prefix of two of the three seeds so only the first seed is loaded.

# Start CARLA server
bash scripts/start_carla.sh

# Evaluate one Bench2Drive route
bash scripts/eval_bench2drive.sh

# Optional: clean CARLA server
bash scripts/clean_carla.sh

Results will be saved to outputs/local_evaluation with the following structure:

outputs/local_evaluation
├── 23687
│   ├── checkpoint_endpoint.json
│   ├── debug_images
│   ├── demo_images
│   └── metric_info.json
├── 23687_debug.mp4
└── 23687_demo.mp4

3. Run expert evaluation

Evaluate expert and collect data

# Start CARLA if not done already
bash scripts/start_carla.sh

# Run expert on one route
bash scripts/run_expert.sh

# Optional: clean CARLA server
bash scripts/clean_carla.sh

Data collected will be stored at data/expert_debug and should have following structure:

data/expert_debug
├── data
│   └── BlockedIntersection
│       └── 999_Rep-1_Town06_13_route0_12_22_22_34_45
│           ├── bboxes
│           ├── depth
│           ├── depth_perturbated
│           ├── hdmap
│           ├── hdmap_perturbated
│           ├── lidar
│           ├── metas
│           ├── radar
│           ├── radar_perturbated
│           ├── results.json
│           ├── rgb
│           ├── rgb_perturbated
│           ├── semantics
│           └── semantics_perturbated
└── results
    └── Town06_13_result.json

Bench2Drive Results

We evaluate TFv6 on the Bench2Drive benchmark, which consists of 220 routes across multiple towns with challenging weather conditions and traffic scenarios.

Method	DS	SR	Merge	Overtake	EmgBrake	Give Way	Traffsign	Venue
TF++ (TFv5)	84.21	67.27	58.75	57.77	83.33	40.00	82.11	ICCV23
SimLingo	85.07	67.27	54.01	57.04	88.33	53.33	82.45	CVPR25
R2SE	86.28	69.54	53.33	61.25	90.00	50.00	84.21	-
HiP-AD	86.77	69.09	50.00	84.44	83.33	40.00	72.10	ICCV25
BridgeDrive	86.87	72.27	63.50	57.77	83.33	40.00	82.11	-
DiffRefiner	87.10	71.40	63.80	60.00	85.00	50.00	86.30	AAAI26
TFv6 (Ours)	95.28	86.80	72.50	97.77	91.66	40.00	89.47	-

DS = Driving Score, SR = Success Rate; Metrics follow the CARLA Leaderboard 2.0 protocol. Higher is better.

Documentation and Resources

For detailed training, data-collection, and large-scale experiment instructions, see the full documentation. In particular, we provide:

• Tutorial Notebooks
• Cross-dataset Training
• Frequently Asked Questions
• Known Issues

We maintain custom forks of CARLA evaluation tools with our modifications:

• scenario_runner_autopilot, leaderboard_autopilot, Bench2Drive, scenario_runner, leaderboard

External Resources

Useful documentations from other repositories:

• CARLA Coordinate Systems
• History of TransFuser
• Common Issues with CARLA
• About Longest6 v2 Benchmark
• About Town13 Benchmark
• Random Scenario Generation
• Manual Scenario Labeling

Other helpful repositories:

• SimLingo, PlanT2, Bench2Drive Leaderboard, Bench2Drive, CaRL

E2E self-driving research:

• Why study self-driving?
• End-to-end Autonomous Driving: Challenges and Frontiers
• Common Mistakes in Benchmarking Autonomous Driving

Acknowledgements

Special thanks to carla_garage for the foundational codebase. We also thank the creators of the numerous open-source projects we use.

• PDM-Lite, leaderboard, scenario_runner, NAVSIM, Waymo Open Dataset

Long Nguyen led development of the project. Kashyap Chitta, Bernhard Jaeger, and Andreas Geiger contributed through technical discussion and advisory feedback.

Citation

If you find this work useful, please consider giving this repository a star ⭐ and citing our work in your research:

@article{Nguyen2025ARXIV,
  title={LEAD: Minimizing Learner-Expert Asymmetry in End-to-End Driving},
  author={Nguyen, Long and Fauth, Micha and Jaeger, Bernhard and Dauner, Daniel and Igl, Maximilian and Geiger, Andreas and Chitta, Kashyap},
  journal={arXiv preprint arXiv:2512.20563},
  year={2025}
}

License

This project is released under the MIT License. See LICENSE for details.