Okta reveals OpenSSL HollowByte flaw that freezes server memory
This digest was compiled by AI from multiple sources — links to the originals are below.
Okta's Red Team disclosed a denial-of-service vulnerability in OpenSSL, dubbed HollowByte, that can freeze server memory with 11-byte TLS requests. The flaw allows an attacker to cause memory fragmentation and exhaustion, potentially leading to a system crash. OpenSSL fixed the issue in June but classified it as a bug, not a vulnerability, issuing no CVE or advisory.
The HollowByte Flaw
OpenSSL versions prior to the June 9 releases — 4.0.1, 3.6.3, 3.5.7, 3.4.6, and 3.0.21 — are vulnerable to a denial-of-service attack using 11-byte TLS handshake messages. The flaw lies in how OpenSSL handles message length declarations: it allocates up to 131 KB of memory per connection based on the attacker's claimed size before verifying the body. This memory is never freed if the connection is dropped, leading to fragmentation.
Memory Fragmentation Impact
Okta's tests on NGINX showed that a 1 GB server was killed by out-of-memory (OOM) after 547 MB of memory was frozen in fragments. On a 16 GB server, HollowByte locked up 25% of system memory without exceeding connection limits. The attack exploits glibc's memory allocator, which holds freed chunks for reuse, causing heap fragmentation that persists after the attack ends.
OpenSSL's Response
OpenSSL's security team classified the issue as a 'bug or hardening' fix, not a vulnerability, and did not assign a CVE or publish an advisory. The patch was merged by Matt Caswell in June with no changelog entry. The Hacker News found no public proof-of-concept exploit as of July 18.
What's Next
Okta has published details but no exploit code. It remains unclear whether other implementations or platforms are similarly affected, and whether OpenSSL will reconsider its classification.
1 source
Okta reveals OpenSSL HollowByte flaw that freezes server memory
