Liquid AI LFM-3B edge deployment on Apple Silicon

Liquid AI utilizes MIT-inspired liquid neural networks to deliver efficient edge computing performance. The Cactus engine achieves 27 tokens per second decode on an M4 Pro CPU using INT8 quantization for local model deployment.

Liquid AI LFM-3B edge deployment on Apple Silicon

Liquid AI secured $250M in Series A funding led by AMD to develop liquid neural networks. This startup, an MIT spin-off, uses models inspired by the computational structure of a worm brain. These models require less computing power and smaller datasets than traditional neural networks. The LFM2 family contains three sizes: 350M, 700M, and 1.2B parameters. The LFM2 architecture uses 16 layers, consisting of 10 double-gated short-range convolution blocks and 6 blocks of grouped query attention. A proprietary neural architecture search engine named STAR optimized these models for target hardware.

The LFM2-24B-A2B model provides a different performance profile for edge deployment. It uses a mixture-of-experts architecture. Liquid AI launched the LFM2-24B-A2B to run on cloud clusters, AI PCs, and mobile devices. The company partners with AMD to maximize performance on Ryzen AI PCs using the RyzenLLM-AI inference engine. Intel supports the model through the OpenVINO toolkit for deployment on AI PCs and edge devices. For local use, developers pull models from the Ollama library or use the LM Studio interface. The Cactus engine achieves 229 tokens per second prefill and 27 tokens per sec decode when running INT8 quantization on an M4 Pro CPU.

Specification LFM2-24B-A2B Detail
Architecture Mixture of Experts
Optimization Engine RyzenLLM-AI, OpenVINO, Cactus
Quantization Support INT8
Local Inference Tools Ollama, LM Studio, Cactus
M4 Pro Prefill Speed 229 tokens/sec
M4 Pro Decode Speed 27 tokens/sec

Liquid AI performance metrics

The d1-3B decision model emits zero tokens. It produces calibrated, typed answers like yes/no or a choice among named options during a single forward pass. The d1-3B model scores 48.57 on the Decision Index v0.2.1 leaderboard, which places it second behind the 12B Winno-12B. It outscores the 36B Decider 35B-A3B by 1.46 points. On seven public text benchmarks, the d1-3B averages 82.9, which exceeds the Decider 4B score of 81.1. The d1-omni-600M variant averages 78.4 on these benchmarks. The d1-3B model processes requests with 8 ms latency on an RTX 4090 and 16 ms on a Jetson AGX Thor.

The LFM2.5-2.6B model uses reinforcement learning for training rather than human preference tuning. It trained on 10 trillion tokens. This model beat the 671B parameter Deepseek R1 on the IFBench instruction-following benchmark. The LFM2.5-2.6B uses ELIV convolutions for short-range reasoning and Grouped Query Attention for long-range logic. This hybrid design allows it to execute tasks with high precision. The model remains available on Hugging Face under the name LiquidAI/LFM-2-2.6B-XP.

Model Category Model Name Key Feature
Decision Model d1-3B 0 output tokens
Decision Model d1-omni-600M Text, image, and audio
Foundation Model LFM-2-2.6B-XP Pure reinforcement learning
Foundation Model LFM2-24B-A2B Mixture of Experts

Local deployment on Mac hardware

Apple Silicon provides the unified memory architecture necessary to run large models locally. An M4 Max with 128GB of RAM runs models that require dedicated GPU workstations on other platforms. The M4 Max provides 546 GB/s of bandwidth. Liquid AI models benefit from this architecture when users deploy them via local backends. The Cactus engine runs on M4 Pro CPUs to achieve specific decoding speeds. Developers use the llama.cpp library to run models with Q4_0 quantization.

The LFM2.5-2.6B model lacks a comparable published API token rate to the Jev-Style 0.8B v3. The LFM2.5-2.6B model also does not fit the specific workload used for the Jev-Style 0.8B v3 in a single request. The Jev-Style 0.8B v3 is not ranked on the public lane for agentic tasks, coding, or reasoning. The LFM2.5-2.6B scores 16.9 in the agentic category on the BenchAlign v5.8 lane, which ranks it #115 out of 131. It scores 12.2 in coding, ranking it #141 out of 152. The LFM2.5-2.6B also scores 26.3 in knowledge, ranking it #171 out of 187. These scores are estimates. The LFM2.5-2.6B does not fit the workload in one request.

The Cactus engine achieves 27 tokens/sec decode on an M4 Pro. This speed depends on the 1k prefill and 100 decode benchmark settings. The M4 Max supports 128GB of unified memory. The M4 Max provides 546 GB/s of bandwidth. Does the M5 Ultra solve the scaling issues seen in previous Ultra generations? Local users run models via Ollama, LM Studio, and mlx-lm. The Anubis OSS app pulls hardware telemetry including GPU utilization and thermal state. Use the Cactus engine for on-device coding agents.

airtrain.ai
airtrain.ai

The airtrain.ai newsroom covers AI research, models and the tools built on them.

More on this topic

Stay ahead of AI

Get the week's most important AI stories delivered to your inbox every Monday.

No spam. Unsubscribe anytime.

More Stories