Skip to content

Latest commit

 

History

History
112 lines (92 loc) · 3.83 KB

prd.md

File metadata and controls

112 lines (92 loc) · 3.83 KB

📘 Product Requirements Document (PRD)

🎯 Project Title

Understanding and Visualizing Optimizers in Machine Learning

👥 Team Members

  • Kien Tran
  • Ken Lam

📅 Timeline

  • Start Date: [Enter Start Date]
  • End Date: [Enter Due Date or Expected Completion Date]
  • Milestones:
    • Research and Notes: [Insert Date]
    • Initial Visualizations: [Insert Date]
    • Interactive Demos Setup: [Insert Date]
    • Final Report and Presentation: [Insert Date]

🧠 Objective

To explore and understand the internal mechanics of widely used optimization algorithms in machine learning, and build clear, intuitive visualizations to demonstrate how each optimizer behaves during training. The goal is to create a resource that helps students and ML practitioners grasp differences and use cases among optimizers.

🧩 Scope of Work

Optimizers to Study

  1. Stochastic Gradient Descent (SGD) with Momentum
  2. RMSProp
  3. Adadelta
  4. AdaGrad
  5. Adam

Each optimizer will be analyzed and compared based on:

  • Update rules and formulas
  • Handling of gradients (magnitude, direction, and adaptation)
  • Performance on convex and non-convex functions
  • Convergence speed and stability

Key Deliverables

  • A clean, readable GitHub repository
  • Python notebooks (or scripts) demonstrating and comparing optimizers
  • Interactive visualizations of optimizer behavior on simple 2D loss surfaces (e.g., Rosenbrock, Himmelblau)
  • Markdown-based educational explanations for each optimizer
  • A final comparative summary table (speed, memory usage, convergence, etc.)
  • Optional: small frontend (e.g., Streamlit or Gradio) to visualize behavior in real-time

🧰 Tech Stack

  • Programming Language: Python
  • Libraries:
    • matplotlib / seaborn for plotting
    • NumPy and PyTorch or TensorFlow for numerical experiments
    • Plotly / mpl-interactions / ipywidgets for interactive visualization
    • Streamlit or Gradio (optional for UI)
  • Project Hosting: GitHub
  • Documentation: Markdown files in the repository (README.md, prd.md, etc.)

🖼️ Visualizations

For each optimizer:

  • 2D contour plot of a loss surface showing optimization path
  • Animation (GIF or interactive widget) showing step-by-step movement
  • Comparison plots of loss vs. epoch
  • Vector field (optional): to show gradient directions vs. update directions

📘 Directory Structure (Suggested)

optimizer-visualization/
│
├── notebooks/
│   ├── adam.ipynb
│   ├── adagrad.ipynb
│   ├── adadelta.ipynb
│   ├── rmsprop.ipynb
│   └── sgd_momentum.ipynb
│
├── visualizations/
│   ├── gifs/
│   └── plots/
│
├── scripts/
│   └── optimizer_classes.py
│
├── static/
│   └── README_assets/
│
├── prd.md
├── README.md
└── requirements.txt

🧪 Evaluation Criteria

  • Accuracy of theoretical explanations and implementation
  • Clarity and usefulness of visualizations
  • Code readability and modularity
  • Engagement of interactive components (if applicable)
  • Final comparative insights across optimizers

🔍 References

💬 Notes for AI Editor

  • Focus on maintaining consistent formatting across notebooks and visualizations
  • Optimize animations for clarity over aesthetics
  • Ensure math formulas are rendered properly in markdown files
  • Suggest tooltip-based explanations for each step in visualization (if using UI)
  • Help with summarizing optimizer comparisons for final presentation/report