whereismyai
Finetuning With Sampling
Finetuning With Sampling is an open source PyTorch implementation for finetuning with sampling and supervised fine-tuning.
This repository provides the official PyTorch implementation for the research paper Finetuning with Sampling by Aayush Karan, Sitan Chen, and Yilun Du from Harvard. It includes scripts to run projection sampling for chemistry and math tasks. Users can execute projection sampling with shard parallelism to generate boosted traces, original solutions, and correctness metrics stored in JSONL files. The project supports traceboosted outputs for MCMC candidate tracking and teacher distillation logits. It also integrates with external codebases for SFT training and evaluation frameworks for model assessment.
Finetuning With Sampling at a glance
Key features
- PyTorch implementation
- Projection sampling scripts
- Shard parallelism support
- Traceboosted JSONL output generation
- Teacher distillation logits tracking
- Model checkpoint evaluation
Use it for
- Run projection sampling for chemistry data
- Run projection sampling for math data
- Generate boosted traces and teacher distillation logits
- Evaluate trained checkpoints on specific tasks
Alternatives to Finetuning With Sampling
- Soup — Fine-tune and align models like Llama-3.1-8B on a 4 GB laptop GPU.
- PEFT — Reduces computational requirements and storage needs for large model training.
- SwanLab — Tracks training metrics and system performance with minimal code integration.
- Floret Torch — Trains word vectors up to twenty times faster than CPU floret.
- ESPnet — Provides reproducible recipes and hundreds of pretrained models for various speech tasks.
- torchao — It speeds up training and inference while reducing memory usage with native PyTorch.
- Stable Baselines3 — Provides reliable, standard PyTorch implementations for reinforcement learning.
- KohakuMossTrainer — Fits adapter training onto cards with 6 GB of VRAM.
- Every number in a tiny GPT — Inspects every lookup, matrix multiplication, and normalisation in a live transformer.
- TensorRT — Speeds up inference by 36X compared to CPU-only platforms.
- SimpleTuner — It supports dozens of model families with advanced memory and enterprise features.
- DeepInfer Studio — Explore AI inference workflows and resource trade-offs without requiring expensive hardware or GPUs.
All Finetuning With Sampling alternatives