whereismyai
Bytesized
Bytesized is a C program that runs SmolLM2-135M and produces identical greedy tokens as Hugging Face transformers on Linux arm64.
Bytesized is a miniature C engine designed to run the SmolLM2-135M language model. It reads original model.safetensors and tokenizer.json files directly without converters. The binary measures 2,946 bytes on Linux arm64, which fits within the storage limit of a single QR code. It uses memory mapping to read weights and performs greedy decoding matching Hugging Face transformers. It targets developers interested in extremely small footprint machine learning execution.
Bytesized at a glance
Key features
- Reads original model.safetensors directly
- Reads tokenizer.json without conversion
- Performs greedy decoding matching Hugging Face
- Maps weights using mmap
- Provides a Python front end called smol
- Includes a reference engine and unpacker
Use it for
- Running a real language model from a QR code
- Executing text continuation on arm64 Linux
- Testing extremely compact LLM engine binaries
Alternatives to Bytesized
- MTPLX — It runs local models twice as fast via native MTP speculative decoding.
- LMDeploy — Accelerates inference with high-performance engines and quantization tools.
- MLX LM — Runs LLMs and fine-tuning natively on Apple silicon hardware.
- AirLLM — Runs massive 70B+ parameter models on small 4GB consumer GPUs.
- LlamaIndex — It provides hosted APIs for parsing documents and extracting typed data with one API key.
- huggingface.js — Connects JavaScript applications directly to Hugging Face models and services.
- whichllm — Ranks models using real benchmarks and hardware fit instead of parameter count.
- TOON — It reduces token usage by 42.6% while maintaining JSON-equivalent accuracy.
- LangChain4j — Integrates LLMs and vector stores directly into Java applications.
- Hugging Face Transformers — It provides a unified API for over 1M+ pretrained model checkpoints on the Hub.
- Soup — Fine-tune and align models like Llama-3.1-8B on a 4 GB laptop GPU.
- FastEmbed — It uses ONNX Runtime for high speed and does not require a GPU or PyTorch.
All Bytesized alternatives