OpenAI’s Jalapeño: A Purpose-Built Inference Chip Aims to Loosen Nvidia’s Grip
OpenAI has entered the semiconductor business with Jalapeño, a custom processor designed specifically for large-language-model inference. Developed with Broadcom and industrialized with Celestica, the accelerator is intended to reduce the company’s direct dependence on Nvidia hardware while making responses from ChatGPT, Codex and the OpenAI API faster and less expensive to serve. 2
The announcement matters because inference is the part of AI computing that users encounter every day. Training builds a model; inference runs that model to produce an answer, generate code or complete an agentic task. As usage grows, the electricity, memory and networking required for millions of individual responses can become as important as the cost of training the model itself.
From software company to full-stack operator
OpenAI unveiled Jalapeño on June 24, 2026, describing it as its first “Intelligence Processor” and the first component of a multi-generation compute platform. The company says it designed the chip from scratch around its understanding of model kernels, memory movement, serving software and product workloads. Broadcom contributed silicon implementation, connectivity and networking technologies, including Tomahawk Ethernet silicon, while Celestica is responsible for board, rack and system integration. 2
The chip is not being positioned as a replacement for every accelerator in OpenAI’s data centers. OpenAI says it will continue to deploy Nvidia and other suppliers for training and inference. Jalapeño is narrower: it is an in-house platform for workloads where latency, energy use and predictable scaling matter more than the broad flexibility of a general-purpose GPU. 1
That distinction is strategically important. Nvidia’s advantage is not only silicon; it also includes a mature software ecosystem, large production capacity and a broad range of products. A custom ASIC must therefore deliver enough real-world value to justify the engineering, manufacturing and software investment required to operate a separate platform.
What Jalapeño is designed to optimize
A language-model request moves through different hardware regimes. During prefill, the system processes the user’s prompt and is mainly limited by computation. During decode, the model generates output token by token and is often limited by memory bandwidth. Agentic workloads add repeated calls, intermediate reasoning and communication between components, so even small delays can compound across a long task.
OpenAI says Jalapeño addresses these bottlenecks through a balanced architecture that keeps model state, including the key-value cache used during generation, close to the compute resources that need it. The design also uses separate communication paths for performance-critical collective traffic and more general data movement. The intended result is less time spent waiting for memory, network transfers or synchronization. 1 3
Tom’s Hardware reports that the accelerator contains 216 GB of HBM4 memory with up to 15.4 TB/s of bandwidth, a 700-watt power rating and peak figures of up to 3.4 MXFP8 PFLOPS and 13.4 MXFP4 PFLOPS. The report also describes 64 core slices, each paired with an HBM slice, and a scale-up design that can connect 128 accelerators in a rack or 2,048 accelerators across a 16-rack pod. 3
The architecture reflects a practical choice: optimize the complete serving system rather than maximize an isolated peak-compute number. Inference hardware can be underused when data arrives late or when one phase of a request is waiting for another. Keeping compute, memory and networking coordinated may produce more useful work per watt even if a competing chip advertises higher theoretical arithmetic throughput.
OpenAI’s first performance results
On August 25, OpenAI published benchmark results using SemiAnalysis’s public InferenceX benchmark. The company compared Jalapeño with commercially available systems across a range of operating points, from high-throughput serving to interactive, low-latency use. OpenAI reports that the chip delivered 1.5 to 1.9 times more AI work per watt at peak throughput and 1.7 to 3.6 times lower end-to-end latency across GPT-OSS 120B, DeepSeek R1 670B and Kimi K2.5 1T. For highly interactive workloads, it reports 2.1 to 4.1 times higher performance. 1
The following figures should be read as company-reported results, not as a universal independent ranking. The comparison depends on model, precision, batch size, power assumptions, software versions and the selected operating point. OpenAI normalized results using published package power and rated Jalapeño at 700 watts; sustained power on the cited tests was at or below 550 watts. 1
Reported measure | OpenAI’s published result | Why it matters |
Peak AI work per watt across three models | 1.5–1.9× | More useful output for a fixed power budget |
End-to-end latency across three models | 1.7–3.6× lower | Faster responses and shorter agent steps |
Interactive-workload performance | 2.1–4.1× higher | More responsive serving at low batch sizes |
Jalapeño package rating | 700 W | A reference point for power-normalized comparisons |
Kimi K2.5 1T peak performance per watt | About 1.5× | Evidence of scaling to a very large public model |
For GPT-OSS 120B, OpenAI reports approximately 1.9 times higher peak mixed tokens per second per kilowatt, 1.7 times lower end-to-end latency and 2.7 times lower minimum time between tokens versus the comparison system. For Kimi K2.5 1T, the company reports about 3.4 times lower end-to-end latency. 1
TechCrunch notes that the comparison was made against an Nvidia Blackwell system and that competing hardware may be newer by the time Jalapeño reaches broad deployment. The publication reports that OpenAI expects very small deployment volumes at the end of 2026, followed by more substantial deployment in 2027. 4 This timing is a crucial caveat: a benchmark advantage today does not automatically predict the economics of a production fleet several quarters later.
AI helped build the chip—and program it
OpenAI says its own models assisted with implementation exploration, verification, circuit optimization and workload analysis, helping the team move from an initial design to tape-out in roughly nine months. The company also says Codex, working with GPT-Astra, helped bring three open-weight models to high performance on Jalapeño within two months. Selected attention and mixture-of-experts blocks reportedly ran 1.5 to 1.8 times faster than existing human-written implementations, although those figures apply to individual blocks rather than complete models. 1
This creates a notable feedback loop. The same class of systems that consumes inference capacity can help engineers design and tune the infrastructure that serves future models. The benefit will depend on verification discipline, chip-design expertise and whether software teams can support a new programming target without losing the productivity provided by Nvidia’s established tools.
What changes for Nvidia—and what does not
Jalapeño could give OpenAI more negotiating leverage, a second source of inference capacity and tighter control over the relationship between models and hardware. If the chip performs as claimed at scale, efficiency gains could lower serving costs, support more simultaneous users and make low-latency agents economically practical. A custom platform may also let OpenAI tune future model architectures around known memory and networking characteristics.
Still, “less dependent” does not mean “independent.” OpenAI’s own announcement says Nvidia and other accelerators will remain part of its infrastructure. Training frontier models requires enormous flexibility and capacity, and a specialized inference ASIC may not suit every model, precision format or research experiment. The practical outcome is more likely to be a heterogeneous fleet in which Jalapeño handles selected serving workloads while GPUs and other accelerators cover training, development and additional inference paths.
The test that matters next
The decisive question is not whether Jalapeño can win a carefully selected benchmark. It is whether OpenAI can manufacture, deploy and operate the platform reliably at gigawatt scale while keeping its software current as models change. That requires production-qualified chips, high-bandwidth memory, networking, rack integration, compiler and kernel support, monitoring, replacement capacity and a smooth path for new model families.
OpenAI says Gen 2 is already deep in development and Gen 3 is taking shape. The company’s stated plan is to begin deploying the first generation inside its compute infrastructure by the end of 2026. 1 If that schedule holds, Jalapeño will represent a shift in OpenAI’s business: from buying most of the underlying acceleration to co-designing a larger portion of the machinery that turns models into services.
The chip’s name may be playful, but the objective is serious. OpenAI wants more control over the cost, speed and availability of inference without abandoning the commercial accelerators that still underpin much of its computing base. Jalapeño is therefore best understood not as a single-product challenge to Nvidia, but as the opening move in a longer hardware strategy—one where model design, serving software, memory, networking and silicon are developed as one system.
References
Comments