The HTTP/2 protocol was introduced in 2015 as a major revision of HTTP/1.1 aimed at improving web performance and security. HTTP/2 provides various optimizations like multiplexing requests over a single connection, header compression, and server push capabilities. While adoption of HTTP/2 has steadily grown over the years, a serious vulnerability dubbed “Rapid Reset” was publicly disclosed in October 2023 that impacts many HTTP/2 implementations.
This post will dive into the details of CVE-2023-44487, explaining how this HTTP/2 vulnerability works and how it is being exploited in the wild to conduct massive denial-of-service (DoS) attacks. We will also cover key steps to mitigate risk from Rapid Reset attacks while vendors issue patches.
The Rapid Reset HTTP/2 Vulnerability Explained
A core capability provided by HTTP/2 is multiplexing, which allows multiple requests and responses to be sent over a single TCP connection. This is accomplished by dividing streams of data into frames, with each stream allocated a unique integer ID. The protocol allows clients to open new streams by sending frame requests with new stream IDs. Streams can be abruptly closed via a RST_STREAM frame specifying the stream ID to terminate.
The Rapid Reset vulnerability arises from the fact that HTTP/2 clients can repeatedly open streams and immediately reset them via RST_STREAM frames in quick succession. Servers typically allocate resources like memory and threads to manage each open stream. A malicious client can easily automate rapid cycles of opening and closing streams to overwhelm the server’s available resources. Even though each stream is promptly canceled, the cumulative effect is to exhaust server capacity leading to denial-of-service.
This vector was exploited in the wild starting in August 2023 to conduct extremely large DDoS attacks. By saturating HTTP/2 implementations with continuous streams of reset frames, attackers were able to take down high-profile sites by exploiting vulnerable server software. Reportedly, these novel HTTP/2 DDoS assaults reached unprecedented sizes upwards of 30 million requests per second.
Summary of CVE-2023-44487
- CVE ID – CVE-2023-44487
- Description – Denial-of-service attacks and key steps to mitigate risk until vendor patches are applied
- CVSS Score – 7.5 (HIGH Vector)
- CVSS Vector – CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H
Vulnerable Products
The following table lists Cisco products that are affected by the vulnerability that is described in this advisory. If a future release date is indicated for software, the date provided represents an estimate based on all information known to Cisco as of the Last Updated date at the top of the advisory. Availability dates are subject to change based on a number of factors, including satisfactory testing results and delivery of other priority features and fixes. If no version or date is listed for an affected component (indicated by a blank field and/or an advisory designation of Interim), Cisco is continuing to evaluate the fix and will update the advisory as additional information becomes available. After the advisory is marked Final, customers should refer to the associated Cisco bug(s) for further details.
| Product | Cisco Bug ID | Fixed Release Availability |
|---|---|---|
| Network and Content Security Devices | ||
| Secure Dynamic Attribute Connector (CSDAC) | CSCwh89890 | 2.2 (Nov 2023)2.3 (Nov 2023) |
| Secure Malware Analytics Appliance, formerly Threat Grid Appliance | CSCwh88721 | 2.19.2 (future release) |
| Network Management and Provisioning | ||
| Business Process Automation | CSCwh88580 | 3.2.003.009 (Nov 2023)4.0.001.003 (Nov 2023)4.0.002.003 (Nov 2023) |
| Crosswork Data Gateway | CSCwh88729 | 4.1.3 (Dec 2023)5.0.2 (Dec 2023)6.0 (Dec 2023) |
| Crosswork Situation Manager (when HTTP/2 is enabled, it’s disabled by default) | CSCwh88658 | |
| Crosswork Zero Touch Provisioning (ZTP) | CSCwh88727 | 6.0.0 (Dec 2023) |
| Data Center Network Manager (DCNM) – SAN Deployments on Windows or Linux | CSCwh88607 | Apply Workaround |
| IoT Field Network Director, formerly Connected Grid Network Management System | CSCwh88604 | 4.11.0 (Dec 2023) |
| Prime Access Registrar | CSCwh88632 | 9.3.3 (Feb 2024) |
| Prime Cable Provisioning | CSCwh91177 | 7.2.1 (Nov 2023) |
| Prime Infrastructure | CSCwh84581 | 3.10.4 (Dec 2023) |
| Prime Network Registrar | CSCwh88631 | 11.2 (Available) |
| Routing and Switching – Enterprise and Service Provider | ||
| IOx Fog Director | CSCwh89927 | 1.22 (Nov 2023) |
| Nexus 3000 Series Switches | CSCwh88614 | |
| Nexus 9000 Series Switches in standalone NX-OS mode | CSCwh88614 | |
| Ultra Cloud Core – Access and Mobility Management Function | CSCwh88572 | 2024.02.0 (May 2024) |
| Ultra Cloud Core – Policy Control Function | CSCwh88574 | 2024.01.0 (Feb 2024) |
| Ultra Cloud Core – Session Management Function | CSCwh88576 | |
| Voice and Unified Communications Devices | ||
| Enterprise Chat and Email | CSCwh88749 | Apply Microsoft Windows Update or Workaround |
| Unified Attendant Console Advanced | CSCwh88736 | Apply Microsoft Windows Update or Workaround |
| Unified Contact Center Domain Manager (CCDM) | CSCwh88737 | Apply Microsoft Windows Update or Workaround |
| Unified Contact Center Enterprise (UCCE) | CSCwh88584 | Apply Microsoft Windows Update or Workaround |
| Unified Contact Center Enterprise – Live Data server | CSCwh88583 | 12.6.2 (Nov 2023) |
| Unified Contact Center Management Portal (CCMP) | CSCwh88737 | Apply Microsoft Windows Update or Workaround |
| Video, Streaming, TelePresence, and Transcoding Devices | ||
| Expressway Series | CSCwh88665 | X14.3.3 (Dec 2023) |
| TelePresence Video Communication Server (VCS) | CSCwh88665 | X14.3.3 (Dec 2023) |
| Wireless | ||
| Connected Mobile Experiences | CSCwh89894 | 11.1 (Feb 2024) |
How Servers are Impacted?
Because support for HTTP/2 is implemented in most major web servers and load balancers, a wide range of products are affected by the Rapid Reset bug. Unfortunately, the stream management logic that hastily allocates resources per stream makes servers susceptible to resource exhaustion via this attack vector.
Notable software impacted includes nginx, Envoy, Apache Tomcat, Caddy, Akamai CDN, and F5 load balancers among many others. Microsoft Azure and Google Cloud products were also confirmed vulnerable. Developers using common HTTP/2 libraries like Netty and nghttp2 are exposed as well. Essentially any product or service relying on affected HTTP/2 implementations could be taken offline by Rapid Reset attacks.
Mitigating HTTP/2 Rapid Reset Attacks
Users are strongly urged to update their software as soon as fixes are available. Several web server moderators released updates. Visit their pages to see more about the mitigations.
- nginx: https://www.nginx.com/blog/http-2-rapid-reset-attack-impacting-f5-nginx-products/
- netty: https://github.com/netty/netty/security/advisories/GHSA-xpw8-rcwv-8f8p
- haproxy: https://www.haproxy.com/blog/haproxy-is-not-affected-by-the-http-2-rapid-reset-attack-cve-2023-44487
- nghttp2: https://github.com/nghttp2/nghttp2/security/advisories/GHSA-vx74-f528-fxqg
- golang: The default stream concurrency limit in golang is 250 streams (requests) per HTTP/2 connection. This value may be adjusted in the golang.org/x/net/http2 package using the Server.MaxConcurrentStreams setting and the ConfigureServer function which are available in golang.org/x/net/http2.
While vendors urgently issue patches, organizations can take steps to mitigate potential HTTP/2 DDoS attacks leveraging Rapid Reset:
- Disable HTTP/2 – The most effective temporary workaround is to completely disable HTTP/2 across your web infrastructure, falling back to HTTP/1.1 only. This prevents exploitation at the cost of losing HTTP/2 performance benefits. Selectively disabling HTTP/2 on less critical servers may be an option.
- Rate limiting – Applying rate limiting policies against HTTP/2 traffic can help curb excessive reset frames. However, high limits may still allow resource exhaustion and attackers will adapt to restrictions.
- Edge filtering – DDoS protection services can try to filter anomalous HTTP/2 traffic though malicious patterns may be difficult to distinguish from legitimate connections.
- Reduce stream limits – Some software allows configuring HTTP/2 stream limits which could minimize resource use, though denial-of-service may still occur.
- Upgrade software – Vendors are rapidly releasing patched versions and configuration guidance to address the root cause. Upgrade priority should go to internet-facing production servers.
Organizations lacking robust DDoS defenses should seriously consider adding services or on-premise solutions to absorb and block HTTP-layer assaults. Slow pipe DoS attacks aimed at bandwidth exhaustion are also a risk.
Protecting Your Infrastructure
Defending against emerging threats like Rapid Reset requires proactive security measures:
- Audit your assets – Inventory web-facing systems and review which server software/versions are used. Prioritize any found vulnerable.
- Follow vendor guidance – Monitor vendor security bulletins and apply recommended mitigations and patches promptly.
- Harden configurations – Disable unused features like server push which may reduce attack surface.
- Employ multilayered defenses – Combine edge filtering, rate limiting, ingress queues, and other countermeasures.
- Monitor closely – Inspect traffic and logs for early signs of exploitation such as abnormal HTTP/2 activity.
- Validate protections – Use tools to fuzz test your HTTP/2 servers for flaws pre and post patching.
- Consider workaround options – Weigh the tradeoffs of disabling HTTP/2 or restricting connectivity versus potential disruption.
With cyber threats inevitably evolving, organizations must take swift action when new vulnerabilities emerge. Adopting proactive security measures and response plans for high-risk scenarios can reduce operational impacts.
Bottom Line
The novel Rapid Reset vulnerability showcases how newly discovered software flaws can be rapidly weaponized to fuel widespread attacks before mitigations are developed. All organizations relying on the ubiquitous HTTP/2 protocol face exposure to this easily exploited denial-of-service vector.
By understanding the mechanics behind this HTTP/2 bug and following recommended patching and workarounds, the risk of disruptive downtime can be substantially reduced. But long term, improving the security posture across the web services ecosystem will require more resilient software and infrastructure designs not vulnerable to resource exhaustion issues.
Extra vigilance and collaboration across the security industry is needed to swiftly identify and eradicate serious defects like Rapid Reset before they are mass exploited. We face an increasingly complex and hostile threat landscape, so the push for improved cyber resilience must persist.