Chip Launch and Core Specifications

OpenAI disclosed the first public benchmarks for Jalapeño, its inaugural inference chip, at the Hot Chips conference on August 25. OpenAI led the architecture design, Broadcom contributed to implementation, networking, and connectivity, while Celestica handled board and rack integration.

Key facts:

• Benchmark disclosure: August 25 at Hot Chips
• Tested version: A0 engineering silicon was used for the public benchmark; B0 has entered fabrication on TSMC’s N3P process
• Rated power: 700W
• Memory: HBM4, with about 15.4 TB/s of single-package memory bandwidth
• Theoretical performance: 13.4 PFLOPS at MXFP4 for the B0 version
• Deployment plan: Very small-scale deployment by the end of 2026, with broader rollout expected in 2027
• Development cycle: Around 9 months from initial design to tape-out

Performance Benchmarks: Jalapeño Versus NVIDIA GB300

Using SemiAnalysis’s InferenceX benchmark, OpenAI compared Jalapeño against NVIDIA systems on three public models: GPT-OSS 120B, DeepSeek R1 670B, and Kimi K2.5 1T. The results show Jalapeño ahead of NVIDIA Blackwell systems on both tokens per user and throughput per kilowatt.

• Peak throughput per watt: Across the three models, Jalapeño delivered 1.5x to 1.9x the throughput of the NVIDIA system
• Low-latency scenario: On DeepSeek R1, Jalapeño reached up to about 700 tokens/s for a single user, versus about 169 tokens/s on NVIDIA GB300, or roughly 4.9x the generation speed
• Kimi K2.5 latency: On the trillion-parameter Kimi K2.5 model, Jalapeño’s lowest end-to-end latency was 1.56 seconds, compared with 5.31 seconds for NVIDIA GB300
• Throughput at fixed service quality: At 100 tokens/s per user, Jalapeño delivered more than 9x the throughput of NVIDIA GB300 on Kimi K2.5

A key architectural difference is that Jalapeño handles both prefill and decode on the same accelerator. NVIDIA Rubin, by contrast, separates prefill onto a dedicated CPX chip. OpenAI’s argument is that real-world traffic mixes change across chat, reasoning models, and agent workloads, so fixed ratios between specialized chips can leave hardware underused.

Accelerated Development: AI in the Chip Pipeline

A high-performance ASIC typically takes at least 18 to 24 months to move from architecture definition through RTL design, verification, physical implementation, and tape-out. Jalapeño reportedly moved from initial design to tape-out in about 9 months.

OpenAI also pointed to AI-assisted work across the chip and software stack. In parts of GPT-OSS attention and MoE modules, AI-generated kernels were 1.5x to 1.8x faster than previous versions written by human experts. With Codex and GPT-Astra, OpenAI completed porting and optimization for DeepSeek R1 and Kimi K2.5 in about two months.

If sustained, this could let model companies feed new model-architecture insights back into hardware design more quickly. Public information also indicates that after the A0 test chip, B0 is already in manufacturing, a second-generation chip is in deep development, and a third generation is being planned.

Architecture Philosophy: Keeping Data Local

In large-model inference, the bottleneck is often not raw compute alone. Moving model weights and KV cache data in and out of memory can dominate performance, leaving general-purpose GPU compute units waiting on data in real workloads.

Jalapeño’s design philosophy can be summarized as reducing data movement:

• Compute cores and HBM4 memory are divided into corresponding slices, giving each core low-latency direct access to its own memory region
• Around 15.4 TB/s of single-package memory bandwidth provides high local data throughput
• Data repeatedly used during generation can be “explicitly placed and kept local,” reducing remote calls and repeated transfers

This is the advantage OpenAI emphasized: Jalapeño aims to deliver higher throughput and lower latency at the same time per unit of power, while traditional systems often trade one against the other.

What Still Needs to Be Proven

The benchmark does not amount to a final victory. The reported numbers were mainly provided by OpenAI, with SemiAnalysis witnessing the run in the lab rather than independently conducting the full test suite. The public results also focus on a single-turn inference workload of roughly 8k input tokens and 1k output tokens. OpenAI has not yet published numbers for longer contexts, multi-turn interactions, or agent workloads.

For now, Jalapeño is best read as an important signal that model companies are moving into inference silicon, not as a final verdict on the GPU ecosystem. The real test will come with actual deployments after late 2026 and broader workload coverage.

Bottom Line

Jalapeño matters beyond its benchmark scores. It shows a leading model company trying to hardwire its model requirements into silicon, shifting from adapting models to available chips toward building chips for models. If the promised cost-per-token benefits hold up in production, AI infrastructure pricing and supply could become more contested.