Pure PyTorch Implementation of NVIDIA paper on Instant Training of Neural Graphics primitives: https://nvlabs.github.io/instant-ngp/

APOLLO: SGD-like Memory, AdamW-level Performance; MLSys'25 Oustanding Paper Honorable Mention

1.5−3.0× lossless training or pre-training speedup. An off-the-shelf, easy-to-implement algorithm for the efficient training of foundation visual backbones.

SlamKit is an open source tool kit for efficient training of SpeechLMs. It was used for "Slamming: Training a Speech Language Model on One GPU in a Day"

Official code for our ECCV'22 paper "A Fast Knowledge Distillation Framework for Visual Recognition"

[arXiv:2309.16669] Code release for "Training a Large Video Model on a Single Machine in a Day"

Can We Find Strong Lottery Tickets in Generative Models? - Official Code (Pytorch)

PyTorch implementation of X3D models with Multigrid training.

[ICLR 2023] "Learning to Grow Pretrained Models for Efficient Transformer Training" by Peihao Wang, Rameswar Panda, Lucas Torroba Hennigen, Philip Greengard, Leonid Karlinsky, Rogerio Feris, David Cox, Zhangyang Wang, Yoon Kim

[ICLR 2023] MLPInit: Embarrassingly Simple GNN Training Acceleration with MLP Initialization

Code for ACL 2025 Main paper "Data Whisperer: Efficient Data Selection for Task-Specific LLM Fine-Tuning via Few-Shot In-Context Learning".

Code for "OnlineAugment: Online Data Augmentation with Less Domain Knowledge" (ECCV 2020)

[CVPR 2020] L2-GCN: Layer-Wise and Learned Efficient Training of Graph Convolutional Networks

[ICLR 2021 Spotlight] "CPT: Efficient Deep Neural Network Training via Cyclic Precision" by Yonggan Fu, Han Guo, Meng Li, Xin Yang, Yining Ding, Vikas Chandra, and Yingyan (Celine) Lin.

✂️ Dataset Culling: Faster training of domain specific models with distillation ✂️ (IEEE ICIP 2019)

[ICLR 2023] Link Prediction with Non-Contrastive Learning

(CVPR 2022) Automated Progressive Learning for Efficient Training of Vision Transformers

This is the official repo for Densely-Anchored Sampling for Deep Metric Learning (ECCV 22).

context denoising training for long-context modeling

[NeurIPS 2020] "FracTrain: Fractionally Squeezing Bit Savings Both Temporally and Spatially for Efficient DNN Training" by Yonggan Fu, Haoran You, Yang Zhao, Yue Wang, Chaojian Li, Kailash Gopalakrishnan, Zhangyang Wang, Yingyan Lin