whereismyai
LMFlow
LMFlow is a toolkit for finetuning and inference of large foundation models.
LMFlow is an extensible toolbox for finetuning large machine learning models, designed to be user-friendly, speedy, and reliable. It supports various training methods including full finetuning, LoRA, QLoRA, and LISA, alongside inference engines like vLLM and SGLang. The platform provides multimodal capabilities for handling images and texts, custom optimizers, conversation templates, and deployment options via Gradio and Flask. It is built for researchers and developers working with large language models who require efficient training and evaluation tools. It runs primarily on Linux and integrates with tools like WandB and LM Evaluation Harness.
LMFlow at a glance
- Best for: Software engineer, Researcher
- Runs on: Linux, self-hosted, open source, CLI
- Pricing: Free
- Released: March 2023
- Category: AI coding tool
- Website: github.com/OptimalScale/LMFlow
Key features
- Finetune large machine learning models
- Support full finetuning, LoRA, QLoRA, and LISA
- Execute vLLM and SGLang-backed inference
- Handle multimodal inputs of images and texts
- Deploy chatbot interfaces with Gradio and Flask
- Apply conversation templates for models
- Track training processes using WandB
Use it for
- Finetuning large foundation models efficiently
- Running multimodal chatbot inference
- Deploying local chatbot user interfaces
- Evaluating open-source chat-style LLMs
Alternatives to LMFlow
- AirLLM — Runs massive 70B+ parameter models on small 4GB consumer GPUs.
- MLflow — Complete observability and evaluation for production AI apps.
- vLLM — Maximizes throughput with PagedAttention and continuous batching.
- LlamaFactory — Provides a code-free CLI and Web UI for training 100+ different LLMs.
- LMDeploy — Accelerates inference with high-performance engines and quantization tools.
- LightLLM — It delivers high-speed inference and serving performance for large language models.
- MLX LM — Runs LLMs and fine-tuning natively on Apple silicon hardware.
- llm-d — Deploys production-grade LLM serving on existing hardware clusters.
- LM Studio — Runs open source language models locally with real-time voice transcription.
- VLMEvalKit — Evaluate large vision-language models across numerous benchmarks with a single command.
- llmware — Build private, local RAG pipelines with 300+ prepackaged models and integrated document parsing.
- Langflow — Builds and deploys AI agents visually with Python and major LLMs.
All LMFlow alternatives