diff --git a/.github/classroom/autograding.json b/.github/classroom/autograding.json
new file mode 100644
index 00000000..fb177874
--- /dev/null
+++ b/.github/classroom/autograding.json
@@ -0,0 +1,34 @@
+{
+ "tests": [
+ {
+ "name": "Task 3.1 - CPU Parallel Operations",
+ "setup": "pip install -e .",
+ "run": "python -m pytest -m task3_1 --tb=no -q",
+ "input": "",
+ "output": "",
+ "comparison": "included",
+ "timeout": 10,
+ "points": 25
+ },
+ {
+ "name": "Task 3.2 - CPU Matrix Multiplication",
+ "setup": "",
+ "run": "python -m pytest -m task3_2 --tb=no -q",
+ "input": "",
+ "output": "",
+ "comparison": "included",
+ "timeout": 10,
+ "points": 25
+ },
+ {
+ "name": "Style Check",
+ "setup": "",
+ "run": "python -m ruff check . && python -m pyright",
+ "input": "",
+ "output": "",
+ "comparison": "included",
+ "timeout": 10,
+ "points": 10
+ }
+ ]
+}
\ No newline at end of file
diff --git a/.github/workflows/classroom.yaml b/.github/workflows/classroom.yaml
new file mode 100644
index 00000000..2853c181
--- /dev/null
+++ b/.github/workflows/classroom.yaml
@@ -0,0 +1,16 @@
+name: GitHub Classroom Workflow
+
+on: [push]
+
+permissions:
+ checks: write
+ actions: read
+ contents: read
+
+jobs:
+ build:
+ name: Autograding
+ runs-on: ubuntu-latest
+ steps:
+ - uses: actions/checkout@v2
+ - uses: education/autograding@v1
\ No newline at end of file
diff --git a/README.md b/README.md
index e82a8886..b589efab 100644
--- a/README.md
+++ b/README.md
@@ -1,32 +1,164 @@
-# MiniTorch Module 3
+# MiniTorch Module 3 - Parallel and GPU Acceleration
-* Docs: https://minitorch.github.io/
+**Documentation:** https://minitorch.github.io/
-* Overview: https://minitorch.github.io/module3.html
+**Overview (Required reading):** https://minitorch.github.io/module3.html
+## Overview
-You will need to modify `tensor_functions.py` slightly in this assignment.
+Module 3 focuses on **optimizing tensor operations** through parallel computing and GPU acceleration. You'll implement CPU parallel operations using Numba and GPU kernels using CUDA, achieving dramatic performance improvements over the sequential tensor backend from Module 2.
-* Tests:
+### Key Learning Goals
+- **CPU Parallelization**: Implement parallel tensor operations with Numba
+- **GPU Programming**: Write CUDA kernels for tensor operations
+- **Performance Optimization**: Achieve significant speedup through hardware acceleration
+- **Matrix Multiplication**: Optimize the most computationally intensive operations
+- **Backend Architecture**: Build multiple computational backends for flexible performance
+## Tasks Overview
+
+| Task | Description
+|---------|-------------
+| **3.1** | CPU Parallel Operations (`fast_ops.py`)
+| **3.2** | CPU Matrix Multiplication (`fast_ops.py`)
+| **3.3** | GPU Operations (`cuda_ops.py`)
+| **3.4** | GPU Matrix Multiplication (`cuda_ops.py`)
+| **3.5** | Performance Evaluation (`run_fast_tensor.py`)
+
+## Documentation
+
+- **[Installation Guide](installation.md)** - Setup instructions including GPU configuration
+- **[Testing Guide](testing.md)** - How to run tests locally and handle GPU requirements
+
+## Quick Start
+
+### 1. Environment Setup
+```bash
+# Clone and navigate to your assignment
+git clone
+cd
+
+# Create virtual environment (recommended)
+conda create --name minitorch python
+conda activate minitorch
+
+# Install dependencies
+pip install -e ".[dev,extra]"
+```
+
+### 2. Sync Previous Module Files
+```bash
+# Sync required files from your Module 2 solution
+python sync_previous_module.py .
+
+# Example:
+python sync_previous_module.py ../Module-2 .
+```
+
+### 3. Run Tests
+```bash
+# CPU tasks (run anywhere)
+pytest -m task3_1 # CPU parallel operations
+pytest -m task3_2 # CPU matrix multiplication
+
+# GPU tasks (require CUDA-compatible GPU)
+pytest -m task3_3 # GPU operations
+pytest -m task3_4 # GPU matrix multiplication
+
+# Style checks
+pre-commit run --all-files
+```
+
+## GPU Setup
+
+### Option 1: Google Colab (Recommended)
+Most students should use Google Colab for GPU tasks:
+
+1. Upload assignment files to Colab
+2. Change runtime to GPU (Runtime → Change runtime type → GPU)
+3. Install packages:
+ ```python
+ !pip install -e ".[dev,extra]"
+ !python -c "import numba.cuda; print('CUDA available:', numba.cuda.is_available())"
+ ```
+
+### Option 2: Local GPU (If you have NVIDIA GPU)
+For students with NVIDIA GPUs and CUDA-compatible hardware:
+
+```bash
+# Install CUDA toolkit
+# Visit: https://developer.nvidia.com/cuda-downloads
+
+# Install GPU packages
+pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
+pip install numba[cuda]
+
+# Verify GPU support
+python -c "import numba.cuda; print('CUDA available:', numba.cuda.is_available())"
```
-python run_tests.py
+
+## Testing Strategy
+
+### CI/CD (GitHub Actions)
+- **Task 3.1**: CPU parallel operations
+- **Task 3.2**: CPU matrix multiplication
+- **Style Check**: Code quality and formatting
+
+### GPU Testing (Colab/Local GPU)
+- **Task 3.3**: GPU operations (use Colab or local NVIDIA GPU)
+- **Task 3.4**: GPU matrix multiplication (use Colab or local NVIDIA GPU)
+
+### Performance Validation
+```bash
+# Compare backend performance
+python project/run_fast_tensor.py # Optimized backends
+python project/run_tensor.py # Basic tensor backend
+python project/run_scalar.py # Scalar baseline
```
-* Note:
+## Development Tools
-Several of the tests for this assignment will only run if you are on a GPU machine and will not
-run on github's test infrastructure. Please follow the instructions to setup up a colab machine
-to run these tests.
+### Code Quality
+```bash
+# Automatic style checking
+pre-commit install
+git commit -m "your changes" # Runs style checks automatically
-This assignment requires the following files from the previous assignments. You can get these by running
+# Manual style checks
+ruff check . # Linting
+ruff format . # Formatting
+pyright . # Type checking
+```
+### Debugging
```bash
-python sync_previous_module.py previous-module-dir current-module-dir
+# Debug Numba JIT issues
+NUMBA_DISABLE_JIT=1 pytest -m task3_1 -v
+
+# Debug CUDA kernels
+NUMBA_CUDA_DEBUG=1 pytest -m task3_3 -v
+
+# Monitor GPU usage
+nvidia-smi -l 1 # Update every second
```
-The files that will be synced are:
+## Implementation Focus
+
+### Task 3.1 & 3.2 (CPU Optimization)
+- Implement `tensor_map`, `tensor_zip`, `tensor_reduce` with Numba parallel loops
+- Optimize matrix multiplication with efficient loop ordering
+- Focus on cache locality and parallel execution patterns
+
+### Task 3.3 & 3.4 (GPU Acceleration)
+- Write CUDA kernels for element-wise operations
+- Implement efficient GPU matrix multiplication with shared memory
+- Optimize thread block organization and memory coalescing
+
+## Important Notes
- minitorch/tensor_data.py minitorch/tensor_functions.py minitorch/tensor_ops.py minitorch/operators.py minitorch/scalar.py minitorch/scalar_functions.py minitorch/module.py minitorch/autodiff.py minitorch/module.py project/run_manual.py project/run_scalar.py project/run_tensor.py minitorch/operators.py minitorch/module.py minitorch/autodiff.py minitorch/tensor.py minitorch/datasets.py minitorch/testing.py minitorch/optim.py
\ No newline at end of file
+- **GPU Limitations**: Tasks 3.3 and 3.4 cannot run in GitHub CI due to hardware requirements
+- **GPU Testing**: Use Google Colab (recommended) or local NVIDIA GPU for GPU tasks
+- **Performance Critical**: Implementations must show measurable speedup over sequential versions
+- **Memory Management**: Be careful with GPU memory allocation and deallocation
diff --git a/installation.md b/installation.md
new file mode 100644
index 00000000..d8924157
--- /dev/null
+++ b/installation.md
@@ -0,0 +1,142 @@
+# MiniTorch Module 3 Installation
+
+MiniTorch requires Python 3.8 or higher. To check your version of Python, run:
+
+```bash
+>>> python --version
+```
+
+We recommend creating a global MiniTorch workspace directory that you will use
+for all modules:
+
+```bash
+>>> mkdir workspace; cd workspace
+```
+
+## Environment Setup
+
+We highly recommend setting up a *virtual environment*. The virtual environment lets you install packages that are only used for your assignments and do not impact the rest of the system.
+
+**Option 1: Anaconda (Recommended)**
+```bash
+>>> conda create --name minitorch python # Run only once
+>>> conda activate minitorch
+>>> conda install llvmlite # For optimization
+```
+
+**Option 2: Venv**
+```bash
+>>> python -m venv venv # Run only once
+>>> source venv/bin/activate
+```
+
+The first line should be run only once, whereas the second needs to be run whenever you open a new terminal to get started for the class. You can tell if it works by checking if your terminal starts with `(minitorch)` or `(venv)`.
+
+## Getting the Code
+
+Each assignment is distributed through a Git repo. Once you accept the assignment from GitHub Classroom, a personal repository under Cornell-Tech-ML will be created for you. You can then clone this repository to start working on your assignment.
+
+```bash
+>>> git clone {{ASSIGNMENT}}
+>>> cd {{ASSIGNMENT}}
+```
+
+## Syncing Previous Module Files
+
+Module 3 requires files from Module 0, Module 1, and Module 2. Sync them using:
+
+```bash
+>>> python sync_previous_module.py
+```
+
+Example:
+```bash
+>>> python sync_previous_module.py ../Module-2 .
+```
+
+Replace `` with the path to your Module 2 directory and `` with `.` for the current directory.
+
+This will copy the following required files:
+- `minitorch/tensor_data.py`
+- `minitorch/tensor_functions.py`
+- `minitorch/tensor_ops.py`
+- `minitorch/operators.py`
+- `minitorch/scalar.py`
+- `minitorch/scalar_functions.py`
+- `minitorch/module.py`
+- `minitorch/autodiff.py`
+- `minitorch/tensor.py`
+- `minitorch/datasets.py`
+- `minitorch/testing.py`
+- `minitorch/optim.py`
+- `project/run_manual.py`
+- `project/run_scalar.py`
+- `project/run_tensor.py`
+
+## Installation
+
+Install all packages in your virtual environment:
+
+```bash
+>>> python -m pip install -e ".[dev,extra]"
+```
+
+## GPU Setup (Required for Tasks 3.3 and 3.4)
+
+Tasks 3.3 and 3.4 require GPU support and won't run on GitHub CI.
+
+### Option 1: Google Colab (Recommended)
+
+Most students should use Google Colab as it provides free GPU access:
+
+1. Upload your assignment files to Colab
+2. Change runtime to GPU (Runtime → Change runtime type → GPU)
+3. Install packages in Colab:
+ ```python
+ !pip install -e ".[dev,extra]"
+ !python -c "import numba.cuda; print('CUDA available:', numba.cuda.is_available())"
+ ```
+
+### Option 2: Local GPU Setup (If you have NVIDIA GPU)
+
+For students with NVIDIA GPUs and CUDA-compatible hardware:
+
+1. **Install CUDA Toolkit**
+ ```bash
+ # Visit: https://developer.nvidia.com/cuda-downloads
+ # Follow instructions for your OS
+ ```
+
+2. **Verify CUDA Installation**
+ ```bash
+ >>> nvcc --version
+ >>> nvidia-smi
+ ```
+
+3. **Install GPU-compatible packages**
+ ```bash
+ >>> pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
+ >>> pip install numba[cuda]
+ ```
+
+## Verification
+
+Make sure everything is installed by running:
+
+```bash
+>>> python -c "import minitorch; print('Success!')"
+```
+
+Verify that the tensor functionality is available:
+
+```bash
+>>> python -c "from minitorch import tensor; print('Module 3 ready!')"
+```
+
+Check if CUDA support is available (for GPU tasks):
+
+```bash
+>>> python -c "import numba.cuda; print('CUDA available:', numba.cuda.is_available())"
+```
+
+You're ready to start Module 3!
\ No newline at end of file
diff --git a/pyproject.toml b/pyproject.toml
index 442ba844..7be5e21d 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -5,17 +5,48 @@ build-backend = "hatchling.build"
[project]
name = "minitorch"
version = "0.5"
+description = "A minimal deep learning library for educational purposes"
+requires-python = ">=3.8"
+dependencies = [
+ "colorama==0.4.6",
+ "hypothesis==6.138.2",
+ "numba==0.61.2",
+ "numpy>=1.24,<2.3",
+ "pytest==8.4.1",
+ "pytest-env==1.1.5",
+ "typing_extensions",
+]
+
+[project.optional-dependencies]
+dev = [
+ "pre-commit==4.3.0",
+]
+extra = [
+ "datasets==2.4.0",
+ "embeddings==0.0.8",
+ "networkx==3.5",
+ "plotly==5.24.1",
+ "pydot==1.4.1",
+ "python-mnist",
+ "streamlit==1.48.1",
+ "streamlit-ace",
+ "torch==2.8.0",
+ "watchdog==1.0.2",
+ "altair==4.2.2",
+]
[tool.pyright]
include = ["**/minitorch"]
-ignore = [
+exclude = [
"**/docs",
- "**/docs/module1/**",
+ "**/docs/module3/**",
"**/assignments",
"**/project",
"**/mt_diagrams",
"**/.*",
"*chainrule.py*",
+ "**/minitorch/autodiff.py",
+ "sync_previous_module.py",
]
venvPath = "."
venv = ".venv"
@@ -30,6 +61,7 @@ reportUnknownLambdaType = "none"
reportIncompatibleMethodOverride = "none"
reportPrivateUsage = "none"
reportMissingParameterType = "error"
+reportMissingImports = "none"
[tool.pytest.ini_options]
@@ -61,7 +93,6 @@ markers = [
"task4_4",
]
[tool.ruff]
-
exclude = [
".git",
"__pycache__",
@@ -72,10 +103,20 @@ exclude = [
"**/mt_diagrams/*",
"**/minitorch/testing.py",
"**/docs/**/*",
+ "minitorch/optim.py",
+ "minitorch/datasets.py",
+ "minitorch/scalar.py",
+ "minitorch/autodiff.py",
+ "minitorch/module.py",
+ "minitorch/tensor.py",
+ "minitorch/tensor_data.py",
+ "minitorch/tensor_functions.py",
+ "minitorch/tensor_ops.py",
+ "sync_previous_module.py",
]
+[tool.ruff.lint]
ignore = [
- "ANN101",
"ANN401",
"N801",
"E203",
@@ -96,7 +137,7 @@ ignore = [
"D107",
"D213",
"ANN204",
- "ANN102",
+ "D203"
]
select = ["D", "E", "F", "N", "ANN"]
fixable = [
@@ -147,5 +188,7 @@ fixable = [
]
unfixable = []
-[tool.ruff.extend-per-file-ignores]
+[tool.ruff.lint.extend-per-file-ignores]
"tests/**/*.py" = ["D"]
+"minitorch/scalar_functions.py" = ["ANN001", "ANN201"]
+"minitorch/tensor_functions.py" = ["ANN001", "ANN201"]
diff --git a/requirements.extra.txt b/requirements.extra.txt
deleted file mode 100644
index 070fa1d0..00000000
--- a/requirements.extra.txt
+++ /dev/null
@@ -1,11 +0,0 @@
-datasets==2.4.0
-embeddings==0.0.8
-plotly==4.14.3
-pydot==1.4.1
-python-mnist
-streamlit==1.12.0
-streamlit-ace
-torch
-watchdog==1.0.2
-altair==4.2.2
-networkx==3.3
diff --git a/requirements.txt b/requirements.txt
deleted file mode 100644
index c9cd8a02..00000000
--- a/requirements.txt
+++ /dev/null
@@ -1,9 +0,0 @@
-colorama==0.4.3
-hypothesis == 6.54
-numba == 0.60
-numpy == 2.0.0
-pre-commit == 2.20.0
-pytest == 8.3.2
-pytest-env
-pytest-runner == 5.2
-typing_extensions
diff --git a/setup.py b/setup.py
deleted file mode 100644
index ff4cfa9f..00000000
--- a/setup.py
+++ /dev/null
@@ -1,3 +0,0 @@
-from setuptools import setup
-
-setup(py_modules=[])
diff --git a/sync_previous_module.py b/sync_previous_module.py
index 9110bf9c..0e1a8bc9 100644
--- a/sync_previous_module.py
+++ b/sync_previous_module.py
@@ -1,50 +1,72 @@
"""
-Description:
-Note: Make sure that both the new and old module files are in same directory!
+Sync Previous Module Files
-This script helps you sync your previous module works with current modules.
-It takes 2 arguments, source_dir_name and destination_dir_name.
-All the files which will be moved are specified in files_to_sync.txt as newline separated strings
+This script helps you sync files from your previous module to the current module.
+It copies files specified in 'files_to_sync.txt' from the source directory to the destination directory.
-Usage: python sync_previous_module.py
+Usage: python sync_previous_module.py
-Ex: python sync_previous_module.py mle-module-0-sauravpanda24 mle-module-1-sauravpanda24
+Examples:
+ python sync_previous_module.py ./my-awesome-module-2 ./my-awesome-module-3
+ python sync_previous_module.py ~/assignments/Module-2-unicorn_ninja ~/assignments/Module-3-unicorn_ninja
"""
import os
import shutil
import sys
-if len(sys.argv) != 3:
- print(
- "Invalid argument count! Please pass source directory and destination directory after the file name"
- )
- sys.exit()
+def print_usage():
+ """Print usage information and examples."""
+ print(__doc__)
-# Get the users path to evaluate the username and root directory
-current_path = os.getcwd()
-grandparent_path = "/".join(current_path.split("/")[:-1])
+def read_files_to_sync():
+ """Read the list of files to sync from files_to_sync.txt"""
+ try:
+ with open("files_to_sync.txt", "r") as f:
+ return f.read().splitlines()
+ except FileNotFoundError:
+ print("Error: files_to_sync.txt not found!")
+ sys.exit(1)
-print("Looking for modules in : ", grandparent_path)
+def sync_files(source, dest, files_to_move):
+ """Copy files from source to destination directory."""
+ if not os.path.exists(source):
+ print(f"Error: Source directory '{source}' does not exist!")
+ sys.exit(1)
-# List of files which we want to move
-f = open("files_to_sync.txt", "r+")
-files_to_move = f.read().splitlines()
-f.close()
+ if not os.path.exists(dest):
+ print(f"Error: Destination directory '{dest}' does not exist!")
+ sys.exit(1)
-# get the source and destination from arguments
-source = sys.argv[1]
-dest = sys.argv[2]
-
-# copy the files from source to destination
-try:
+ copied_files = 0
for file in files_to_move:
- print(f"Moving file : ", file)
- shutil.copy(
- os.path.join(grandparent_path, source, file),
- os.path.join(grandparent_path, dest, file),
- )
- print(f"Finished moving {len(files_to_move)} files")
-except Exception as e:
- print(
- "Something went wrong! please check if the source and destination folders are present in same folder"
- )
+ source_path = os.path.join(source, file)
+ dest_path = os.path.join(dest, file)
+
+ if not os.path.exists(source_path):
+ print(f"Warning: File '{file}' not found in source directory, skipping")
+ continue
+
+ try:
+ os.makedirs(os.path.dirname(dest_path), exist_ok=True)
+ shutil.copy(source_path, dest_path)
+ print(f"Copied: {file}")
+ copied_files += 1
+ except Exception as e:
+ print(f"Error copying '{file}': {e}")
+
+ print(f"Finished copying {copied_files} files")
+
+def main():
+ if len(sys.argv) != 3:
+ print("Error: Invalid number of arguments!")
+ print_usage()
+ sys.exit(1)
+
+ source = sys.argv[1]
+ dest = sys.argv[2]
+ files_to_move = read_files_to_sync()
+
+ sync_files(source, dest, files_to_move)
+
+if __name__ == "__main__":
+ main()
diff --git a/testing.md b/testing.md
new file mode 100644
index 00000000..5f9ecff1
--- /dev/null
+++ b/testing.md
@@ -0,0 +1,129 @@
+## Testing Your Implementation
+
+### Running Tests
+
+This project uses pytest for testing. Tests are organized by task:
+
+```bash
+# Run all tests for a specific task
+pytest -m task3_1 # CPU parallel operations
+pytest -m task3_2 # CPU matrix multiplication
+pytest -m task3_3 # GPU operations (requires CUDA)
+pytest -m task3_4 # GPU matrix multiplication (requires CUDA)
+
+# Run all tests
+pytest
+
+# Run tests with verbose output
+pytest -v
+
+# Run a specific test file
+pytest tests/test_tensor_general.py # All optimized tensor tests
+
+# Run a specific test function
+pytest tests/test_tensor_general.py::test_one_args -k "fast"
+pytest tests/test_tensor_general.py::test_matrix_multiply
+```
+
+### GPU Testing Strategy
+
+**CI Limitations:**
+- GitHub Actions CI only runs tasks 3.1 and 3.2 (CPU only)
+- Tasks 3.3 and 3.4 require local GPU or Google Colab
+
+**Option 1: Google Colab Testing (Recommended):**
+```python
+# In Colab notebook
+!pip install -e ".[dev,extra]"
+!python -m pytest -m task3_3 -v
+!python -m pytest -m task3_4 -v
+!python -c "import numba.cuda; print('CUDA available:', numba.cuda.is_available())"
+```
+
+**Option 2: Local GPU Testing (If you have NVIDIA GPU):**
+```bash
+# Verify CUDA is available
+python -c "import numba.cuda; print('CUDA available:', numba.cuda.is_available())"
+
+# Test GPU tasks locally
+pytest -m task3_3 # GPU operations
+pytest -m task3_4 # GPU matrix multiplication
+
+# Debug GPU issues
+NUMBA_DISABLE_JIT=1 pytest -m task3_3 -v # Disable JIT for debugging
+```
+
+### Style and Code Quality Checks
+
+This project enforces code style and quality using several tools:
+
+```bash
+# Run all pre-commit hooks (recommended)
+pre-commit run --all-files
+
+# Individual style checks:
+ruff check . # Linting (style, imports, docstrings)
+ruff format . # Code formatting
+pyright . # Type checking
+```
+
+### Task 3.5 - Performance Evaluation
+
+**Training Scripts:**
+```bash
+# Run optimized training (CPU parallel)
+python project/run_fast_tensor.py
+
+# Compare with previous implementations
+python project/run_tensor.py # Basic tensor implementation
+python project/run_scalar.py # Scalar implementation
+```
+
+### Pre-commit Hooks (Automatic Style Checking)
+
+The project uses pre-commit hooks that run automatically before each commit:
+
+```bash
+# Install pre-commit hooks (one-time setup)
+pre-commit install
+
+# Now style checks run automatically on every commit
+git commit -m "your message" # Will run style checks first
+```
+
+### Debugging Tools
+
+**Numba Debugging:**
+```bash
+# Disable JIT compilation for debugging
+NUMBA_DISABLE_JIT=1 pytest -m task3_1 -v
+
+# Enable Numba debugging output
+NUMBA_DEBUG=1 python project/run_fast_tensor.py
+```
+
+**CUDA Debugging:**
+```bash
+# Check CUDA device properties
+python -c "import numba.cuda; print(numba.cuda.gpus)"
+
+# Monitor GPU memory usage
+nvidia-smi -l 1 # Update every second
+
+# Debug CUDA kernel launches
+NUMBA_CUDA_DEBUG=1 python -m pytest -m task3_3 -v
+```
+
+**Performance Profiling:**
+```bash
+# Time specific operations
+python -c "
+import time
+import minitorch
+backend = minitorch.TensorBackend(minitorch.FastOps)
+# Time your operations here
+"
+
+# Profile memory usage
+python -m memory_profiler project/run_fast_tensor.py
+```
\ No newline at end of file