whereismyai
Sulci Oss
Sulci Oss is a Python library that caches LLM responses by semantic meaning to reduce cost and latency.
Sulci Cache is an AI native context-aware semantic caching library for LLM applications. It caches language model responses by semantic meaning rather than exact string matches, returning cached answers instantly for similar queries. It works with backends such as SQLite, ChromaDB, FAISS, Qdrant, Redis, Milvus Lite, and Sulci Cloud. It includes integrations for LangChain and LlamaIndex, an MCP server, a zero-code-change caching proxy compatible with OpenAI and Anthropic, and an asynchronous wrapper. It targets developers building LLM apps and agentic AI systems who want to avoid paying for identical answers twice.
Sulci Oss at a glance
- Best for: Software engineer
- Runs on: Python, CLI, self-hosted
- Pricing: Free
- Released: October 2026
- Category: AI coding tool
- Website: github.com/sulci-io/sulci-oss
Key features
- Caches LLM responses by semantic meaning
- Supports SQLite, ChromaDB, FAISS, Qdrant, Redis, and Milvus backends
- Offers a zero-code-change OpenAI and Anthropic caching proxy
- Provides integrations for LangChain and LlamaIndex
- Includes a Model Context Protocol server
- Features a non-blocking asynchronous wrapper
- Blends conversation context into lookups
Use it for
- Caching repeated user questions in LLM applications
- Reducing API costs for agentic AI workflows
- Speeding up response times for similar queries
- Managing cache layers via a proxy without code changes
Alternatives to Sulci Oss
- GPTCache — It decreases expenses and enhances performance by caching LLM query results.
- LMCache — It stores, compresses, searches, and reuses KV caches across clusters.
- oMLX — It uses paged SSD KV caching to reduce agent TTFT to under 5 seconds.
- llm-d — Deploys production-grade LLM serving on existing hardware clusters.
- LangChain4j — Integrates LLMs and vector stores directly into Java applications.
- Codebase Memory MCP — High-speed indexing and persistent knowledge graphs for AI coding agents.
- AI Memory — Shared memory for coding agents that works across machines and team members.
- Rubi — It cuts AI token waste by keeping sessions short and mapping project knowledge.
- claude-mem — It shares persistent, searchable agent memory across sessions and devices.
- Infinity — Achieves 0.1 milliseconds query latency on million-scale vector datasets.
- Headroom — Reduces token usage by 60 to 95 percent for JSON and lowers costs while maintaining the same answers.
- Lcc — Cuts token counts significantly without losing exact session details.
All Sulci Oss alternatives