Foundry Local

Foundry Local is an on-device AI inference tool that runs models locally with complete data privacy.

Foundry Local allows developers to run AI models directly on user devices for complete data privacy without requiring an Azure subscription. The product provides native SDKs to download, cache, load, and call optimized local models in-process. It handles hardware execution providers automatically so applications can target NPU, GPU, or CPU without custom device plumbing. The runtime and cached models remain local to support offline app execution. Supported languages include Python, JavaScript, C#, and Rust, with an optional OpenAI-compatible server available for frameworks like LangChain.

Foundry Local at a glance

Best for: Software engineer
Runs on: Web, CLI
Pricing: Free
Released: March 2025
Category: AI coding tool
Website: foundrylocal.ai

Key features

Native SDKs for Python, JavaScript, C#, and Rust
In-process local model inference
Automatic hardware targeting for NPU, GPU, and CPU
Offline app runtime with cached models
Optional OpenAI-compatible server for frameworks like LangChain
Complete data privacy for prompts, audio, and responses

Use it for

Build local AI capabilities into applications
Run AI inference offline without a network connection
Target hardware execution providers without custom plumbing

Alternatives to Foundry Local

Local Code Community — Runs local offline models and built-in tools securely inside an app sandbox.
Shimmy — Runs local AI models with zero Python or C++ dependencies.
fal — Access 1,000+ generative models and high-performance serverless GPUs.
Strata — Runs large AI models locally on consumer gaming PCs without server reliance.
Lumabri — Runs large AI models across multiple computers by sharing memory and layer workloads.
SmallCode — Runs fully local with budget-managed context and a forgiving parser for small models.
ODS — Automates complex local AI infrastructure setups with one command.
Flare Local Inference — Optimizes memory and bandwidth for workstations and compact systems.
OpenCode — Works with 75+ model providers, local models, and existing subscriptions
MNN — It delivers high-performance inference and training with a lightweight footprint.
Union.ai — Runs machine learning tasks natively on Apple Silicon GPUs using Metal.
oMLX — It uses paged SSD KV caching to reduce agent TTFT to under 5 seconds.

All Foundry Local alternatives