LLM CLI

LLM CLI: LLM is a command-line utility and Python library for interacting with and managing various large language models.

LLM serves as a CLI tool and Python library for working with a wide range of large language models, including OpenAI, Claude, Gemini, Qwen, Gemma, Kimi, DeepSeek, and Mistral. Users can interact with models via remote APIs or install plugins to execute local models on their own machine. The tool supports tasks such as running prompts, managing interactive chats, generating embeddings, and extracting structured content from text, images, audio, and video. All prompts and responses are automatically logged to a local SQLite database for future reference. It also provides a plugin architecture to extend functionality, including custom tools and template loaders.

LLM CLI at a glance

Best for: Software engineer
Runs on: CLI, Python
Pricing: Free
Released: April 2023
Category: AI coding tool
Website: llm.datasette.io

Key features

Execute prompts directly from the command-line interface
Run interactive chat sessions with models
Log prompts and responses in a local SQLite database
Generate and manage embeddings and collections
Extract structured data from unstructured content using schemas
Integrate local models via plugins
Attach images, audio, and video files to prompts
Define custom model aliases and prompt templates

Use it for

Interacting with multiple AI models from a terminal
Building local search engines using embeddings
Extracting structured information from raw text or media
Automating LLM tasks within Python scripts
Managing history of LLM conversations

Companions

Tools that articles about LLM CLI recommend using with it, each with its job.

Cline — Open source, any model, with approval on every step
AnythingLLM — Runs fully on-device with no accounts, API keys, or token limits required.
Open WebUI — ChatGPT-style UI and knowledge bases on top of local models.
CrewAI — Point CrewAI's agents at the local endpoint.

Alternatives to LLM CLI

llmspy.org — Self-host access to 530+ AI models across 24 providers with built-in tools.
LLM Workflow Engine — Manage and execute complex LLM workflows directly from the command line.
LiteLLM — One OpenAI-format API for 100+ LLM providers
LangChain4j — Integrates LLMs and vector stores directly into Java applications.
WebLLM — Runs high-performance language models directly inside web browsers.
LM Studio — Runs open source language models locally with real-time voice transcription.
MLX LM — Runs LLMs and fine-tuning natively on Apple silicon hardware.
LightLLM — It delivers high-speed inference and serving performance for large language models.
MLC LLM — Enables native optimization and deployment of AI models across platforms.
llm-d — Deploys production-grade LLM serving on existing hardware clusters.
LMQL — It enables modularized, constraint-based prompting with portable code.
llmware — Build private, local RAG pipelines with 300+ prepackaged models and integrated document parsing.

All LLM CLI alternatives