The architecture of local processing
The 20B parameter on-device model on the latest Apple silicon delivers impressive results for local AI tasks. Apple uses a hybrid architecture to balance privacy and power across its ecosystem. On-device components utilize approximately 3 billion parameters to run on chips like the A17 Pro or M-series processors. For more demanding workloads, the 20B parameter model employs an exotic architecture where a small model predicts which experts to load from Nand into RAM. This system selects one set of experts per query rather than switching experts for every token. This method differs from a typical Mixture of Experts approach by performing the selection once per query. You need at least 8GB of RAM and an A17 Pro chip to run Apple Intelligence, something you should know if you plan to upgrade soon. The system uses a family of five foundation models to power these experiences. This architecture integrates processing and memory within one platform. This integration simplifies capacity planning for users. Concurrent use of both engines allows the system to achieve much higher throughput. For example, testing on an M4 Max with macOS 26.6.2 shows that short, validated, single-question requests on the Apple Neural Engine take 9.9 ms, while the MLX GPU takes 12.2 ms.
Performance and hardware limits
I find the efficiency of these on-device models very convincing. While the 3 billion parameter model outperforms Microsoft’s Phi-3-mini, Mistral-7B, Google’s Gemma-7B, and Meta’s Llama-3-8B in internal benchmarks, it still lacks the necessary reasoning depth found in much larger server-based systems. These on-device components provide faster response times and lower power consumption, although the reduced parameter count potentially limits cognitive capabilities compared to larger systems. The 4K context size restricts the amount of data the system processes at one time. When I tested the Apple Foundation Advanced model on an iPhone 18 Pro, the device identified ramen and sliced pork in a photo immediately. I also saw the system refuse to identify famous people like John Turnus or Tim Cook. ReALM, another Apple AI system, converts visual screen layouts into textual representations to understand on-screen elements. ReALM achieved performance gains of over 5% for on-screen reference resolution compared to existing systems, even though its smallest model has fewer parameters than GPT-4.
| Model/Device | Minimum Chip | Minimum RAM | AI Capability |
|---|---|---|---|
| iPhone 15 Pro/Max | A17 Pro | 8GB | Legacy AI |
| iPhone 16 Pro/Max | A18 Pro | 8GB | Native AI |
| iPhone 17 Pro Max | A19 Pro | 12GB | Agentic AI |
The iPhone 17 Pro Max uses the A19 Pro chip and 12GB of RAM to support advanced Agentic AI. In contrast, the iPhone 16 Pro and 16 Pro Max use the A18 Pro chip with 8GB of RAM for the first full native AI-optimized architecture. For users with an iPhone 15 Pro or Max, the A17 Pro chip provides 8GB of RAM for first-generation AI compatibility. The 2025 upgraded models also included better tool-use capabilities, improved reasoning, and support for 15 different languages. While the on-device models excel in efficiency, the server-based models remain comparable to older systems like OpenAI’s GPT-3.5 and Meta’s Llama-3-70B. The gap between the on-device model and server-based models becomes clear when users require advanced visual reasoning or complex mathematical problem-solving. The on-device model handles basic tasks like summarizing missed messages or rewriting emails, but it cannot match the absolute performance levels of advanced cloud-based systems.
Private Cloud and developer constraints
Apple handles complex requests through Private Cloud Compute when on-device models lack the necessary computational power. When you initiate a task, a model running on your device analyzes whether the task can be completed on-device first. If a larger, server-based model requires more power, Apple Intelligence sends your request to Private Cloud Compute. This server-based system uses Apple Foundation Models trained with proprietary data and refined using data from Google frontier models. Apple uses no Gemini models, no Google Assistant code, and no Google Search infrastructure. The cloud models rely on proprietary data via RL for training, which they then refine with data from Google frontier models. Private Cloud Compute processes information in secure enclaves and does not store results permanently. Users can see how their data is processed by turning on transparency logging in Settings under Privacy and Security. This report shows Apple Intelligence activity for the user.
Developers using Private Cloud Compute face specific restrictions. Apple limits access to developers with fewer than 2 million first-time app downloads from any of their apps. This restriction applies to the developer account as a whole rather than a single application. I find these limits create a barrier for new developers attempting to scale their AI features. The system handles text, images, and audio through these diverse models. Can the system handle such high-volume requests without hitting developer download limits?
