DeepSeek + Pi Outperforms Claude Code? Pi Founder: This Pairing Was Anticipated
DeepSeek + Pi Outperforms Claude Code? Pi Founder: This Combination Was Foreseen
On August 11, Pi Harness founder Mario Zechner shared data from developer 0xEvan, who processed nearly one billion input tokens through Pi on DeepSeek V4 Flash with a 99.93% cache hit rate, at a cost of $2.65. Uncached, the same workload was projected to cost $132. [IMAGE:0]
That same day, developer Shantanu Goel noted that DeepSeek V4 Flash's cache hit rate ranged from 94% to 97% across other harnesses, yet consistently exceeded 99% within Pi. 'This is particularly valuable with local models,' Zechner observed. [IMAGE:1]
Additional cases of this kind have accumulated across the community. [IMAGE:2]
The results also call to mind Mario's May assessment of the Pi–DeepSeek V4 pairing: 'pi + ds4 == sovereign AI enterprise ready clearly.' Paraphrased for clarity: 'Pi + DeepSeek 4 — enterprise AI, achieved.' [IMAGE:3]
At the time, the remark was a casual aside offered after observing a developer run terminal Tetris on Pi with DeepSeek V4. Three months later, a public benchmark supplied the missing data.
Has the Pi–DeepSeek pairing actually outperformed Claude Code?
Composio, a tooling provider for AI agents, recently conducted a public benchmark in which a single model — DeepSeek V4 Flash — was executed across eight agent harnesses on 30 demanding agentic tasks. Each task required the agent to act, invoke tools, and finish the work autonomously.
First place went to Pi Agent, which completed 20 of 30 tasks (66.7%). Oh My Pi followed with 17; Claude Code, Codex, and Deep Agents each completed 16. Prime Agent completed 15, though six of its runs were excluded — two due to evaluator timeouts and four with no recorded output. Hermes Agent also completed 15; OpenCode finished last with 14. [IMAGE:4]
Holding the model constant, a change of harness alone lifted success from 46.7% to 66.7% — a 20-percentage-point swing.
The cost differential was more pronounced. Pi averaged $0.028 per successful task; Claude Code required $0.195 — approximately seven times higher. [IMAGE:5]
Pi's median completion time of 132.2 seconds trailed Claude Code (122.7s) and OpenCode (129.7s), yet on a combined basis of success, speed, and cost, it produced the round's strongest result. [IMAGE:6]
The test demonstrates a 'harness multiplier effect': surrounding tooling amplifies or diminishes a model's realized performance. With the correct harness, the same model becomes more reliable and more efficient; with the wrong one, task success and throughput decline measurably even when underlying intelligence is unchanged.
Composio accordingly argues against isolated model evaluation: an agent leaderboard that names a model without disclosing its harness yields an incomplete score.
Why the minimalist Pi prevailed
One further detail merits attention: Pi was deployed as a clean, unmodified default installation, with only the MCP server plugins required by the benchmark attached. No custom settings, tuning, or bespoke configuration was applied. This near-out-of-the-box configuration passed the most tasks.
Prime Agent, by contrast, generated the benchmark's largest sessions — up to 3.5 million tokens and 33 tool calls per session. The agent, in effect, compiled a task list the length of a phone book before commencing actual work.
Session sizes were such that the evaluator timed out during processing alone. Two runs were unscorable and four left no trace, excluding six runs in total. Among valid runs, Prime's pass count matched Hermes' while its duration approached twice that of Pi.
The results present a clear inversion: Prime, the most feature- and session-heavy harness, was slowed by its own weight, while the leaner Pi passed the most tasks at lower overhead. Within this benchmark, added layers yielded no measurable benefit.
The findings challenge the premise that more configuration yields better outcomes. The default instinct — select the largest model, layer on every plugin, extension, and feature — rests on the assumption that capability scales with complexity. This benchmark suggests an alternative: pair a fast, low-cost model with a clean, lightweight harness, and validate the combination on real tasks.
DeepSeek V4 Flash, true to its name, is a flash-tier model optimized for speed and efficiency rather than intelligence-benchmark supremacy. Pi's lightweight configuration prevailed for a simple reason: each added layer introduces another point of failure, each added tool another decision, each oversized instruction file more noise to process before acting.
A clean harness affords the model a direct path from task receipt to completion; an overbuilt harness induces detours. The default installation prevailed precisely because the path was shorter — and with it, the opportunity for error.
How was a 99.9% cache hit rate achieved?
Pi is not a DeepSeek-specific harness but an open agent foundation. Through extensions, developers may rewrite the system prompt, filter conversation history, customize context compression, and add, remove, or toggle tools at runtime; the final request payload is even inspectable and editable before transmission. Such programmability creates substantial latitude for DeepSeek cache optimization.
The DeepSeek API caches prompt prefixes. When the opening token sequence of a subsequent request matches its predecessor exactly, the server serves those tokens from cache and bills them well below standard input rates. Hit pricing is materially lower than miss pricing.
The critical constraint: this is prefix caching, matched from the first token onward. Any change at the front of the context can invalidate a substantial tail of previously cached tokens. The earlier the mutation, the larger the collateral.
A typical agent request comprises system prompt, tool definitions, conversation history, and current-turn additions. Each step re-transmits a large body of prior context. Longer conversations thus repeat more content and, in principle, favor caching. However, a harness that reorders timestamps, tool sequences, or history summaries on each turn defeats stable reuse regardless of context length.
A crop of harness projects dedicated to DeepSeek cache optimization has emerged. Reasonix, an open-source terminal coding agent built around DeepSeek prefix caching, has attracted wide developer attention. One developer, preferring Pi, built DeepPi to transplant Reasonix's methods into that harness, reporting stable cache hit rates of 99.7%–99.9% on DeepSeek API calls. [IMAGE:7]
I ported a portion of Reasonix's performance advantages into a Deepseek-optimized package for Pi. It engages only when the Deepseek API is in use; once active, cache hit rates have held steady between 99.7% and 99.9%.
Reasonix's governing principles: stabilize the context head, append rather than mutate, and minimize the cost of change.
In implementation, Reasonix injects a minimal, stable environment summary at startup, never regenerating it per turn. Stale tool outputs are truncated and pruned before summary compression, preventing a large cat result from twenty turns prior from persisting in the prefix. Built-in tool Schema contracts are documented and subjected to regression review on change: unannounced reshuffling of tool definitions invalidates the cache with no outward indication.
In dual-model mode, the execution and planning models operate in separate, cache-stable sessions, never interleaved within a single context — the project's most elegant design decision. Inserting planning turns into the same conversation, the obvious shortcut, would destabilize both roles' caches; isolated sessions preserve each prefix.
Comparable third-party extensions have emerged throughout the Pi ecosystem. pi-deepseek-cache (https://pi.dev/packages/@rohaquinlop/pi-deepseek-cache), for instance, pursues the same core objective and shares substantial design DNA with Reasonix.
Where Reasonix insists on a stable environment summary injected at startup, pi-deepseek-cache's P0 layer does the same: freezing date and working directory at launch, thereby eliminating cache invalidations triggered by dynamic defaults such as `Current date: YYYY-MM-DD` and `Current working directory:
Reasonix's 'prune stale tool outputs before compression' mandate is implemented in pi-deepseek-cache's P3 layer through cache-friendly compression: when history exceeds summarization thresholds, deepseek-v4-flash at temperature 0 produces deterministic summaries, which are hash-cached such that identical history always reuses byte-identical summaries — shielding the prefix from summarization variance.
Reasonix's further precept — documented tool Schema contracts with regression review on change — finds its analogue in pi-deepseek-cache's P2 layer: SHA-256 prefix diagnostics trace when the prefix shifts, enabling developers to pinpoint invalidation root causes. [IMAGE:8]
The cost impact is readily quantified: deepseek-v4-flash input pricing falls from $0.14 to $0.003 per million tokens (−98%); deepseek-v4-pro falls from $3.00 to $0.025 (−99%).
Conclusion
Notably, DeepSeek has not yet shipped its own harness. On August 11, a WeChat official account registered under 'DeepSeek Harness Team' — widely interpreted as a signal of an impending product launch. Private testing has commenced; a public release is expected shortly.
What would an official harness add? The principal answer is 'native integration.' Third parties are limited to reverse-engineering the public API; an official team can coordinate directly with model developers — tuning the model to the harness's invocation patterns and drawing on non-public model internals. Such integration depth is unattainable by any third party.