Meet Bonsai: The First 27B AI Mannequin That Suits on Your Cellphone – Decrypt

I fashions eat up a whole lot of reminiscence. A 27-billion-parameter AI mannequin, thought-about medium-sized by business requirements, wants roughly 54 GB of reminiscence to run on half precision. Most laptops cannot maintain that. Some desktop rigs cannot both.

Earlier this week, PrismML launched one at 3.9 GB—sufficiently small to suit on an iPhone.

Parameters are the variety of dials and tweaks a mannequin can deal with. The extra parameters, the denser and extra succesful a mannequin is.

Bonsai 27B is the primary 27B-class mannequin to clear the reminiscence ceiling of a shopper smartphone, operating at 11 tokens per second on an iPhone 17 Professional Max. (Tokens are the essential unit of data that AI fashions can deal with and produce.) The ternary variant, at 5.9 GB, hits round 26 tokens per second on an M5 Professional laptop computer. Each are free below Apache 2.0.

The compression methodology, constructed on Caltech mental property, reduces every mannequin weight from 16 bits of floating-point precision to a single signal—+1 or -1 within the binary construct, one in all three values within the ternary. Every group of 128 weights shares a 16-bit scaling issue, touchdown the binary variant at 1.125 bits per weight: 14 occasions smaller than the full-precision authentic. The ternary mannequin provides a zero state for barely extra expressive energy and settles at 1.71 bits.

In simpler phrases, this implies a ternary AI mannequin makes use of solely three settings for every inner worth—detrimental, zero, or optimistic—whereas a regular AI can select from about 65,000 settings.

PrismML did that with out shedding a lot of the output high quality.

What makes this completely different from standard “low-bit” fashions is that nothing will get a higher-precision escape hatch: embeddings, consideration, and the total language mannequin head are all compressed end-to-end. Most quantized builds hold sure delicate layers at full precision, which finally ends up rising their measurement as a tradeoff for higher high quality. Bonsai does not play that sport.

That is the second main launch within the household. In March, PrismML shipped Bonsai 8B, a 1.15 GB mannequin that proved the 1-bit structure may survive at 8 billion parameters with out its reasoning collapsing. The soar to 27 billion is the place the stakes change—that scale is the place sustained chain-of-thought reasoning, dependable software use, and multi-step agentic habits really emerge persistently—the issues smaller fashions nonetheless fumble.

Benchmarks

Throughout 15 benchmarks evaluated in pondering mode on NVIDIA H100 GPUs—spanning information, math, coding, and power use—Ternary Bonsai 27B averages 80.49, or 94.6% of the full-precision mannequin. The 1-bit variant hits 76.11.

General, on benchmarks, the fashions carry out a lot better than Gemma 4 or Qwen 3.6 when it comes to how a lot potential they provide for his or her measurement.

The fashions are fairly good for what they provide, and contemplating how little sources they require, they take small {hardware} (smartphones and decrease finish PCs) to a different stage when it comes to capabilities. AIME25 and AIME26, modeled on the American Invitational Arithmetic Examination, are available in 93.7% for Ternary Bonsai 27B versus 95.3% for the a lot greater Qwen 3.6B. Bonsai scores 86 factors in codig vs 88 for Qwen 3.6 and 77% on common information vs 83 for Qwen 3.6.

The mannequin additionally makes use of a hybrid consideration spine the place roughly 75% of the layers are linear quite than full quadratic consideration. That structure is what makes a 262K-token context window sensible on-device—one thing a regular consideration stack would make prohibitively costly on cellphone {hardware}.

We examined it

We ran Bonsai 27B ourselves. Coding takes iteration: single-shot prompts will not compete with cloud frontier fashions. Being native and free makes that irrelevant. For our Zombie Sort sport—a first-person typing-horror browser sport—two vibe coding rounds produced clear collision detection, correct scoring logic, and graphics that held collectively. The mannequin grasps construction early; the second cross refines quite than rebuilds.

Apparently sufficient, some fashions (just like the skeletons) appeared extra elaborate than those from GPT 5.6 Sol. It doesn’t imply it’s higher by any means, simply that on this process it produced a cute skeleton whereas the AI king made a poorer stylistic alternative.

The sport is accessible for testing right here.

Inventive writing is a extra certified story, and the factors is extra subjective.

Roughly talking, the outcomes aren’t significantly imaginative you probably have a zero-shot immediate in thoughts.

That mentioned, Bonsai produces tales with constant inner logic, pacing, and arc—higher, or on par with Claude Haiku and even Sonnet on decrease effort on comparable prompts. For a mannequin that runs fully by yourself {hardware} with no API prices, that is quite a bit to say.

The story it created might be present in our Github repository.

PrismML additionally ships a DSpark speculative decoding layer alongside the mannequin—a light-weight drafter that proposes blocks of candidate tokens, which the principle mannequin verifies in a single ahead cross quite than producing token-by-token. On an H100 that provides a 1.37x throughput enhance with no change in output high quality, since verification preserves the precise output distribution. On Apple Silicon it isn’t but enabled by default, however for GPU serving it is an actual achieve.

Apple’s curiosity provides a business dimension. PrismML CEO Babak Hassibi confirmed to CNBC that the corporate is in early talks with Apple, which is evaluating the compression expertise for potential on-device use.

Hassibi mentioned a compressed Gemma mannequin is subsequent within the pipeline, adopted by bigger frontier fashions; 1-bit Bonsai 27B is accessible without spending a dime obtain now below Apache 2.0. Should you want a primer on operating fashions like this domestically, try our information.