Briefly
- CAISI’s analysis ranked DeepSeek V4 Professional eight months behind the U.S. frontier, utilizing an IRT-based scoring system throughout 9 benchmarks together with two non-public, unverifiable datasets.
- The associated fee comparability excluded all U.S. fashions deemed too costly or too weak—leaving solely GPT-5.4 mini, in opposition to which DeepSeek was nonetheless cheaper on 5 out of seven benchmarks.
- Stanford’s 2026 AI Index discovered the U.S.-China efficiency hole on public leaderboards had collapsed to 2.7%.
A U.S. authorities institute printed its verdict on China’s strongest AI: eight months behind, and the extra time passes, the broader the hole will get. The web learn the methodology and began asking questions.
CAISI—the Heart for AI Requirements and Innovation, a unit inside NIST—launched its analysis of DeepSeek V4 Professional on Could 1. The conclusion: DeepSeek’s open-weight flagship “lags behind the frontier by about 8 months.”
CAISI additionally calls it essentially the most succesful Chinese language AI mannequin it has evaluated thus far.
The scoring system
CAISI would not common benchmark scores like most evaluators do. As an alternative, it applies Merchandise Response Idea—a statistical technique from standardized testing—to estimate every mannequin’s latent functionality by monitoring which issues it solves and which it would not, throughout 9 benchmarks in 5 domains: cybersecurity, software program engineering, pure sciences, summary reasoning, and math.
The IRT-estimated Elo scores: GPT-5.5 at 1,260 factors, Anthropic’s Claude Opus 4.6 at 999. DeepSeek V4 Professional scores round 800 (±28), which may be very near GPT-5.4 mini at 749. In CAISI’s system, DeepSeek sits nearer to the outdated technology of GPT mini than to Opus.
The factors system in benchmarks rating fashions the way in which standardized checks rating college students—not by uncooked share appropriate, however by weighting which issues they remedy and which they miss, producing a factors estimate that solely means one thing relative to different fashions in the identical analysis. The extra factors, the higher the mannequin is generally phrases, with the most effective mannequin’s rating changing into the reference level to see how succesful a mannequin is.
It’s unimaginable to breed CAISI’s outcomes as a result of two of the 9 benchmarks are personal, and in these two benchmarks is the place the hole is widest. For instance, GPT-5.5 scored 71% on CTF-Archive-Diamond, one among CAISI’s cybersecurity checks with DeepSeek registering round 32%.
On public benchmarks, the image shifts. GPQA-Diamond—PhD-level science reasoning, scored as share appropriate—positioned DeepSeek at 90%, one level behind Opus 4.6’s 91%. Math olympiad benchmarks (OTIS-AIME-2025, PUMaC 2024, SMT 2025) put DeepSeek at 97%, 96%, and 96%. On SWE-Bench Verified—actual GitHub bug fixes, scored as share resolved—DeepSeek scored 74% to GPT-5.5’s 81%. DeepSeek’s personal technical report claims V4 Professional matches Opus 4.6 and GPT-5.4.
For price comparability, CAISI filtered out any U.S, mannequin that carried out considerably worse or price considerably extra per token than DeepSeek. Just one mannequin cleared the bar: GPT-5.4 mini. That is the whole U.S. frontier, filtered to a single entry.
DeepSeek got here out cheaper on 5 of seven benchmarks even beating OpenAI’s tiniest and least succesful AI mannequin.
The counterargument: Is the hole greater or smaller?
Criticizing CAISI’s methodology would not totally vindicate DeepSeek. The AI developer underneath the pseudonym Ex0bit pushed again immediately: “There is not any ‘hole’, and nobody’s 8 months behind. We have been trolled on each closed U.S drop and flexed on with open weights.”
The Synthetic Evaluation Intelligence Index v4.0—a ranking system monitoring frontier mannequin intelligence throughout 10 evaluations—reveals OpenAI close to 60 factors and DeepSeek within the low 50s as of Could 2026, compressed far tighter than a 12 months in the past.
Primarily based on standardized benchmarks, their methodology reveals the hole is definitely getting smaller.
When DeepSeek first emerged in January 2025, the query was whether or not China had already caught up. U.S. labs scrambled to reply. Stanford’s 2026 AI Index—launched April 13—stories the Enviornment leaderboard hole between Claude Opus 4.6 and China’s Dola-Seed-2.0 Preview is shrinking, separated now by solely 2.7%.
CAISI plans to launch a fuller IRT methodology write up within the close to future.
Day by day Debrief E-newsletter
Begin daily with the highest information tales proper now, plus authentic options, a podcast, movies and extra.