DNN Pipeline Template

A starter repository for deep‐learning projects, providing a standardized environment, launcher script, and tooling for debugging & profiling.

---

Table of Contents

1. Features
2. Prerequisites
3. Project Setup
4. Usage
   • Launching a Script
   • Debug Mode
   • Profiling Mode
5. Directory Layout
6. Contributing
7. License

---

Features

• Portable launcher (start.py) isolates your system environment
• PYTHONPATH is automatically set to project root
• Single entry point for scripts, debug, and profiling
• Git LFS support for large model/checkpoint files

---

Prerequisites

• Ubuntu 24.04 (other OS not officially supported)
• Python 3.11+
• Git with Git LFS

---

Project Setup

# 1. Clone & fetch LFS objects
git clone https://github.com/<your-org>/<your-repo>.git
cd <your-repo>
git lfs install
git lfs pull

# 2. Create & activate virtualenv, install dependencies
./install_env.sh

---

Usage

All project scripts are launched via the start.py wrapper:

./start.py -- /path/to/your_script.py [--script-args]

• start.py ensures a clean virtualenv and sets PYTHONPATH to the repo root
• Use -- to separate start.py options from your script’s arguments

Launching a Script

./start.py -- scripts/train.py -- --config configs/train.yaml

Debug Mode

Waits for a VS Code debug client to attach on port 5678:

./start.py -d -- scripts/train.py -- --config configs/train.yaml

1. Press F5 in VS Code
2. Select the “Attach to start.py” debug configuration

Profiling Mode

Profiles with Scalene, output to prof.json:

./start.py -p -- scripts/train.py -- --config configs/train.yaml

---

Directory Layout

├── install_env.sh       # creates virtualenv, installs dependencies
├── start.py             # launcher + debug/profiler wrapper
├── requirements.txt     # Python dependencies
├── scripts/             # user scripts (train.py, eval.py, etc.)
├── configs/             # YAML/JSON config files
├── src/                 # project code (models/, data/, utils/)
└── prof.json            # latest Scalene profile output

---

Contributing

1. Fork the repo
2. Create a feature branch (git checkout -b feat/my-feature)
3. Commit your changes
4. Open a pull request against main

Please follow PEP 8, include tests in tests/, and ensure all CI checks pass.

---

License

This project is licensed under the MIT License – see the LICENSE file for details.