Apple Unveils Core AI Framework, Tailoring On-Device Generative AI for Proprietary Silicon

Frontiers Jun 29, 2026

At WWDC 2026, Apple released Core AI, the official successor to Core ML. The framework enables developers to run large language models and generative AI entirely on-device, with support for custom converted PyTorch models and pre-optimized open-source models.

Apple stated that Core AI delivers a unified architecture capable of deploying vision models as small as 3 billion parameters and inference models up to 70 billion parameters across iPhone, iPad, Mac, and Apple Vision Pro.

Core AI underpins Apple Intelligence. As Apple rolls out its next-generation operating systems and developer toolchains, the framework allows developers to build what Apple terms 'custom intelligence features.' Core AI runs exclusively on Apple Silicon, guaranteeing user data privacy, eliminating server dependency, and incurring no per-token cloud expenses.

Core AI's key capabilities include: unified hardware access that lets workloads run seamlessly across CPU, GPU, and Neural Engine through a single API; memory-safe Swift APIs that enable zero-copy data paths and fine-grained inference memory control; and ahead-of-time (AOT) compilation, which shifts pre-processing off-device to achieve near-instant model loading.

As noted, PyTorch models can be converted to Core AI models via Core AI PyTorch. The simplest method is to export PyTorch as torch.export.ExportedProgram, then convert it to a CoreAI AIProgram using TorchConverter().add_exported_program(ep).to_coreai().

Alternatively, developers can leverage the library's built-in composite operators — including attention mechanisms, RoPE embeddings, RMSNorm, and gather-matmul — to construct new Core AI models from existing PyTorch models, register custom lowering functions that map novel PyTorch operators to Core AI IR, or author custom Metal kernels for deeper, low-level optimization.

A critical step in converting PyTorch models is compressed deployment targeting Apple hardware. The process applies optimizations such as quantization and palletization, which are natively aligned with Core AI runtime execution modes to guarantee efficient on-device performance.

Model compression reduces memory footprint — including both disk size and runtime usage — lowers inference latency, cuts power consumption, or achieves all of these optimizations simultaneously.

A defining characteristic of AIModel execution is automatic specialization for the current hardware and operating system version. This occurs upon the model's initial load into the model cache; consequently, first-time usage may take longer than subsequent invocations. Developers can customize SpecializationOptions, inspect or delete cached models through AICacheModel, and share the model cache across app groups.

With Core AI's introduction, Apple provides three distinct approaches for ML and AI on its operating systems: Core ML, Core AI, and MLX Swift. Per developer discussions on Hacker News, Apple's recommended usage is: Core ML for 'classic non-neural network ML' — decision trees or tabular feature engineering; Core AI for neural networks and Transformers; and MLX for custom model weights, albeit with potentially lower performance. Community feedback further indicated that while Core AI 'makes integrating high-performance LLMs easier,' its long-term trajectory hinges on 'official Core AI/community development going forward.'