Exploring the Edge AI Trend with Liquid AI’s LFM2.5‑2.6B Model

In the wind-lashed cab of a service truck with no cell bars, a technician asks an on-device assistant to diagnose a pressure anomaly. The agent plans steps, calls a tool for schema lookup, reasons across logs—and returns an answer before the engine light flashes. No cloud. No wait. No data leaving the cab.

TL;DR

• LFM2.5‑2.6B is a 2.6‑billion‑parameter agentic model trained on 34T tokens, designed to run privately on phones and CPUs with sub‑2.5 GB footprint, enabling low‑latency tool use, planning, and multi-step workflows entirely on device.
• It outperforms larger models in instruction following and tool use, delivers up to 220 tokens/sec on Apple Silicon CPUs, and scales to 15,000 tokens/sec on H100 GPUs at high concurrency—ideal for field ops, healthcare, and regulated sectors.
• Challenges include tighter memory budgets and slightly lower coding proficiency than larger LLMs; mitigations include smart tool orchestration, prompt budgeting, and lightweight distillation or fine‑tuning. Teams can follow an on-device rollout playbook for fast pilot-to-production transitions.

What is LFM2.5‑2.6B and why does it matter?

LFM2.5‑2.6B is a compact, 2.6‑billion‑parameter agentic model trained on 34 trillion tokens and purpose-built for on-device planning, tool calling, and multi-step reasoning. It runs efficiently on smartphones and CPUs for private, low-latency inference, often matching or beating much larger models in instruction following and tool use—without cloud dependencies or per‑call API costs.

Purpose-built for agent workflows, LFM2.5‑2.6B emphasizes planning, external tool interaction, and multi-turn tasks. Its doubled 128K vocabulary supports non‑Latin scripts and dense, long inputs common in agent loops. Despite its size, the model frequently rivals models up to 4× larger on instruction-following and tool-use benchmarks, with a known tradeoff: it trails larger LLMs slightly on coding tasks. For teams designing secure edge assistants, our primer on private inference best practices explains how to align privacy, latency, and cost goals with on-device deployment.

How does a small model behave like a big one?

A layered training pipeline blends supervised instruction-following, domain-specialist “teacher” models, efficient distillation, and agentic reinforcement learning—so the 2.6B‑parameter student learns to plan, call tools, and reason robustly in real environments. The result is a single, versatile on-device model tuned for instruction following, tool use, and long-context workflows.

Under the hood:

• Tokenizer and context: A 128K‑entry tokenizer with in‑place extension improves handling non‑Latin scripts and long inputs essential for agents managing multi-doc contexts.
• Supervised fine-tuning (SFT): Broad coverage first, then targeted skill building in reasoning, tool use, and search.
• Specialist teachers: Domain experts (instruction following, math, knowledge, code, tools, long context) are optimized—then ensembled.
• Multi‑Domain On‑Policy Distillation (MOPD): The student learns from multiple teachers in one model, accelerating convergence and preserving cross-domain competence.
• Agentic RL: Training occurs within real multi-turn, tool‑centric environments, with rubric checks, programmatic rewards, and safety gates ensuring consistent, useful behavior.

If you’re assembling an edge agent stack, our edge AI playbook breaks down how to balance model size, tokenizer strategy, and tool orchestration.

How fast is LFM2.5‑2.6B on phones, CPUs, and GPUs?

On-device, LFM2.5‑2.6B decodes up to 220 tokens/sec on modern Apple Silicon and ~113 tokens/sec on recent CPU‑only PCs, with a memory footprint under 2.5 GB—fast enough for real-time assistants without cloud calls. On H100 GPUs, it reaches ~15,000 tokens/sec at high concurrency, processing about 1.3B tokens/day per GPU.

Key performance notes:

• Edge formats: Compatible with llama.cpp GGUF, MLX, and vLLM; optimized for Apple Silicon, general CPUs, and mobile devices.
• Speed and footprint: ~220 tok/s (M5 Max CPU), ~113 tok/s (Ryzen AI Max CPU), <2.5 GB.
• Data center scaling: ~15,000 output tok/s per H100 at high concurrency; ~1.3B tokens/day on a single GPU.
• Practical takeaway: You can prototype on a laptop and deploy to a modest device fleet—then burst to GPUs for batch workloads, all using the same model family.

Why edge agents change the economics of private workflows

On-device agents deliver sub‑second interactions, data minimization by default, and predictable costs without per‑call cloud billing. For regulated sectors, keeping PHI, PII, or sensitive telemetry local reduces exposure and simplifies compliance. For field ops, offline availability and fast tool use raise first‑time‑fix rates while lowering retruck costs.

• Privacy and compliance: Sensitive data remains local; audit scopes shrink and retention risks drop.
• Latency: Planning, calling tools, and reasoning happen in real time—no round trips.
• Cost control: Avoid egress, storage, and per‑token API charges; leverage existing CPUs.
• Reliability: Works in low‑connectivity or air‑gapped settings with consistent performance.
  For planning templates and checklists, see our deployment guides and worksheets.

Where does it shine? Real‑world scenarios

LFM2.5‑2.6B’s strengths—tool use, instruction following, and efficient reasoning—map cleanly to high‑value edge cases where privacy and uptime matter.

• Field operations: A technician’s tablet runs diagnostics, parses sensor logs, and executes stepwise procedures through local tools—even miles from a cell tower.
• Healthcare: A clinic workstation summarizes local notes, plans intake workflows, and drafts discharge instructions, all without PHI leaving the device.
• Regulated finance/public sector: An on‑prem agent triages cases, queries internal systems, and assembles investigative packets under strict data-handling rules.

At‑a‑glance: Fit for task

What are the challenges—and how do you mitigate them?

Edge constraints—memory, thermals, and intermittent connectivity—mean careful prompt budgeting, selective caching, and conservative tool chains. While LFM2.5‑2.6B trails bigger models on coding, augmenting with targeted tools and lightweight fine‑tuning narrows the gap. Fleet MLOps (updates, evals, rollback) is essential for reliable rollouts.

Mitigation playbook:

• Prompt budgets: Cap context windows; compress history; summarize between turns.
• Tooling first: Offload retrieval, math, parsing, and validation to local tools.
• Skill boosts: Use small, domain‑specific fine‑tunes or distillation for coding-heavy tasks.
• Eval harness: Track task success, latency, and safety flags per release; enforce rollback.
• Device health: Set thermal/CPU guards; stage downloads; verify quantization compatibility.
  Our checklists for private edge deployments outline validation gates from pilot to production.

How to deploy an on‑device agent in 6 steps

You can move from prototype to a pilot fleet in days by standardizing formats, adopting a tool-first design, and instrumenting evaluation from day one.

1. Choose your format and quantization

• Start with GGUF (llama.cpp) or MLX for Apple Silicon; verify sub‑2.5 GB targets.
• Benchmark token/sec and memory under realistic prompts.

2. Define the tool belt

• Add local retrieval, calculators, schema/intake checkers, and validators.
• Treat the model as planner/controller; tools do the heavy lifting.

3. Implement an OpenAI‑compatible endpoint

• Serve locally or on‑prem so clients integrate quickly.
• Enforce per‑tool safety and schema validation.

4. Engineer prompts for edge

• Use concise system prompts; summarize histories; cap token budgets.
• Add “plan, act, reflect” scaffolds for reliability.

5. Instrument evaluations and safety

• Measure task success, latency, and tool call accuracy.
• Gate releases with programmatic checks and guardrails.

6. Pilot, then scale

• Start with 10–50 devices; monitor thermals and battery.
• Roll updates gradually; keep rollback paths ready via version pins.
  For templates and scripts, explore our implementation resources.