LFM2.5-Encoders for Fast Long-Context Inference on CPU

The Evolution of Efficient Encoders

LiquidAI's release of LFM2.5-Encoder-230M and LFM2.5-Encoder-350M marks a significant step in the evolution of efficient natural language processing models. Building on the foundation laid by models like BERT and the more recent ModernBERT, these new encoders are designed to push the boundaries of accuracy, speed, and context length, particularly for CPU-based inference. The core innovation lies in their ability to handle document-scale inputs—up to 8,192 tokens—with latency that grows slowly, making them ideal for continuous, high-volume tasks that typically run on less specialized hardware.

Unlike their LFM2.5-Retrievers predecessors, which were optimized for multilingual search, the LFM2.5-Encoders are general-purpose. They are pre-trained with a masked-language objective, allowing them to be fine-tuned for a broader array of applications, including classification, token-level tasks, and search. This versatility addresses a critical need in production NLP, where applications like intent routers, safety filters, and text classifiers require robust, always-on performance without incurring prohibitive costs.

Architectural Innovations and Performance

The LFM2.5-Encoders are initialized from their respective LFM2 decoder backbones (LFM2.5-230M and LFM2.5-350M) and transformed into bidirectional encoders through several key modifications. These include the implementation of a bidirectional attention mask, allowing each token to consider context from both sides, and non-causal short convolutions that symmetrically mix in neighboring tokens. Masked language modeling, where 30% of tokens are masked during training, further enhances their learning capability. The training process itself is bifurcated into two stages: an initial phase for general language competence on a large web corpus with a 1,024-token context, followed by a long-context adaptation phase extending to 8,192 tokens, which strengthens factual, legal, and multilingual understanding.

Benchmark results underscore the impressive performance of these models. The LFM2.5-Encoder-350M, despite its relatively small size, ranks fourth among 14 models across GLUE, SuperGLUE, and multilingual classification tasks, outperforming models nearly ten times its size. The LFM2.5-Encoder-230M also surpasses ModernBERT-base and all EuroBERT models while being smaller than most. Crucially, their inference speed on CPU is a standout feature; the LFM2.5-Encoder-230M is approximately 3.7 times faster than ModernBERT-base at 8,192 tokens, processing a full contract or transcript in under 30 seconds on a laptop CPU. While GPU performance also shows gains for long inputs, the dramatic CPU advantage is where these encoders truly shine.

Practical Applications and Accessibility

The practical implications of the LFM2.5-Encoders are substantial, particularly for developers and organizations seeking to deploy advanced NLP solutions without significant hardware investment. The article highlights several live demos, including zero-shot prompt routing, policy linting, spell checking, and PII detection, all running efficiently in CPU-only Hugging Face spaces. These applications demonstrate the models' capability to handle complex text understanding tasks with high throughput and low cost.

For high-volume tasks such as classification, routing, extraction, or scoring, a fine-tuned LFM2.5-Encoder presents a more economical and faster alternative to larger generative LLMs. Their ability to fit and run effectively on existing CPU hardware democratizes access to powerful AI tools, enabling a broader range of businesses and developers to integrate sophisticated NLP into their workflows. The choice between the 230M and 350M versions allows users to prioritize either higher accuracy or tighter hardware constraints and throughput, offering flexibility for diverse deployment scenarios.