OpenAI says Jalapeno chip outperforms Nvidia's GB300 in inference tests
This digest was compiled by AI from multiple sources — links to the originals are below.
OpenAI said its new Jalapeno chip outperformed Nvidia's GB300 in two key inference metrics during testing. The chip, developed with Broadcom, will begin supporting OpenAI's AI models later this year. The announcement came as OpenAI presented first benchmarks at the Hot Chips conference.
Key Facts
- Jalapeno led in AI work per unit of power and response speed against Nvidia's GB300.
- OpenAI plans to start using Jalapeno to support its AI models later this year.
- Jalapeno delivers 1.5x to 1.9x more AI work per watt at peak throughput across three tested models.
- The chip was tested using SemiAnalysis's public InferenceX benchmark, with some runs verified on-site.
- Jalapeno uses TSMC's N3P node, HBM4 memory, and 15.4 TB/s memory bandwidth per package.
Benchmark Results
Jalapeno outperformed Nvidia's GB300 in AI work per unit of power and response speed. OpenAI chip chief Richard Ho said the chip achieved strong results at 700 watts, reducing data center power costs. Across three tested models, Jalapeno delivered 1.5x to 1.9x more AI work per watt at peak throughput. End-to-end latency was 1.7x to 3.6x lower than the best commercially available systems. On GPT-OSS 120B, Jalapeno reached about 1,400 tokens per second per user; on Deepseek R1 670B, over 700 tokens per second.
Chip Architecture
Jalapeno pairs a single reticle-sized compute die on TSMC's N3P node with an N3E I/O chiplet and HBM4 memory. The package delivers 15.4 TB/s of memory bandwidth, with cores and memory divided into matching slices. Jalapeno uses MXFP numerical formats, compressing AI math data down to 4-bit in MXFP4. The chip is designed only for inference, not for training AI models.
Deployment Plans
OpenAI will determine which of its AI models will run on Jalapeno, allowing customers to choose cost savings or better performance. The company plans to start using the new chips to support its AI models later this year. Jalapeno was developed in partnership with Broadcom, announced last year and completed in record time.
4 sources
OpenAI says Jalapeno chip outperforms Nvidia's GB300 in inference tests
