Univo

Univo is a Python data deduplication library that finds duplicates and compares structured records locally.

Univo is a Python library for entity resolution and dataset deduplication that processes text datasets and structured records. It combines fuzzy matching, local embeddings, and field-level rules to find duplicate records without requiring servers, training, or API keys. Users define schemas to specify which fields to compare and set rules that rule out matches regardless of similarity scores. The software supports spelling, meaning, and exact values, utilizing a 15 MB multilingual default model that runs offline or any Model2Vec model from Hugging Face or a folder. It is designed for developers and data practitioners who need to clean data, deduplicate machine learning datasets, and run real-time duplicate checks.

Univo at a glance

Best for: Software engineer, Researcher
Runs on: API
Pricing: Free
Released: October 2026
Category: AI productivity tool
Website: github.com/leopoldch/univo

Key features

Field-level comparison for dictionaries, lists, and nested fields
Rule-based filtering to prevent invalid matches
Spelling, meaning, and exact value matching
Offline operation with a 15 MB multilingual model
Support for Model2Vec models from Hugging Face or folders
Index management for checking new records as they arrive
Saved and loaded snapshots for indexes

Use it for

Data cleaning and dataset deduplication
Entity resolution across structured records
Machine learning dataset preparation
Real-time duplicate checks for incoming records

Alternatives to Univo

Docling — Provides advanced PDF understanding and local execution for sensitive data.
MinerU — Parses complex documents and tables into structured markdown for agents.
Quello — Ask across Notion, GitHub, Google Drive, and PDFs with exact-source citations.
Unstract — It uses dual LLMs to eliminate hallucinations and ensure extraction accuracy.
deepdoctection — Orchestrates layout analysis, OCR, and model fine-tuning in PyTorch pipelines.
Univer — Integrates high-performance office tools and AI agents into your application.
Remio — Local-first AI memory indexing unlimited files and meetings with complete data control.
Xberg — Extracts documents, code, images, and audio using one native-Rust engine.
MarkItDown — Converts documents into LLM-friendly Markdown for better token efficiency.
Sparrow — Extracts structured data locally from documents without cloud dependencies.
Dragomand — Provides shared, offline machine translation for any Linux application.
SwarmVault — Compiles raw sources into a local-first knowledge graph without requiring cloud access.

All Univo alternatives