Skip to content

Latest commit

 

History

History
198 lines (178 loc) · 9.83 KB

README.md

File metadata and controls

198 lines (178 loc) · 9.83 KB
Raw

The HTTP Garden

The HTTP Garden is a collection of HTTP servers and proxies configured to be composable, along with scripts to interact with them in a way that makes finding vulnerabilities much much easier. For some cool demos of the vulnerabilities that you can find with the HTTP Garden, check out our ShmooCon 2024 talk.

Acknowledgements

We'd like to thank our friends at Galois, Trail of Bits, Narf Industries, and Dartmouth College for making this project possible.

This material is based upon work supported by the Defense Advanced Research Projects Agency (DARPA) under contract number HR0011-19-C-0076.

Getting Started

Dependencies

  1. The HTTP Garden runs on x86_64 Linux, and is untested on other platforms.
  2. The target servers are built and run in Docker containers, so you'll need Docker.
  3. You'll also need the following Python packages, which you can get from PyPI (i.e. with pip) or from your system package manager:
  • docker
    • For interacting with Docker
  • pyyaml
    • For parsing yaml
  • tqdm
    • For progress bars

If you're installing Python packages with your system package manager, be aware that the package names may need to be prefixed with py3-, python3-, or python-, depending on the system.

  1. I also highly recommend installing rlwrap from your package manager, because it makes the Garden repl a whole lot more fun.

Building

  • Build the base image:
docker build ./images/http-garden-soil -t http-garden-soil

This image contains some basic utilities, plus a forked AFL++ that facilitates collecting coverage from processes without killing them.

  • Build some HTTP servers and proxies:
docker compose build gunicorn hyper nginx haproxy nginx_proxy

There are, of course, way more targets in the HTTP garden than the ones we just built. It's just that building them all takes a long time. Even building these few will take a few minutes!

Running

  • Start up some servers and proxies:
docker compose up gunicorn hyper nginx haproxy nginx_proxy
  • Start the repl:
rlwrap python3 tools/repl.py
  • Filter a basic GET request through HAProxy, then through an Nginx reverse proxy, then send the result to Gunicorn, Hyper, and Nginx origin servers, and display whether their interpretations match:
garden> payload 'GET / HTTP/1.1\r\nHost: whatever\r\n\r\n' # Set the payload
garden> transduce haproxy nginx_proxy # Run the payload through the reverse proxies
[1]: 'GET / HTTP/1.1\r\nHost: whatever\r\n\r\n'
    ⬇️ haproxy
[2]: 'GET / HTTP/1.1\r\nhost: whatever\r\n\r\n'
    ⬇️ nginx_proxy
[3]: 'GET / HTTP/1.1\r\nHost: echo\r\nConnection: close\r\n\r\n'
garden> servers gunicorn hyper nginx # Select the servers
garden> grid # Show their interpretations
         g
         u
         n
         i h n
         c y g
         o p i
         r e n
         n r x
        +-----
gunicorn|✓ ✓ ✓
hyper   |  ✓ ✓
nginx   |    ✓

Seems like they all agree. Let's try a more interesting payload:

garden> payload 'POST / HTTP/1.1\r\nHost: a\r\nTransfer-Encoding: chunked\r\n\r\n0\n\r\n'
garden> grid
         g
         u
         n
         i h n
         c y g
         o p i
         r e n
         n r x
        +-----
gunicorn|✓ ✓ X
hyper   |  ✓ X
nginx   |    ✓

There's a discrepancy! This is because Nginx supports \n as a line ending in chunk lines, but Hyper and Gunicorn don't. Nginx is technically violating RFC 9112 here, but the impact is likely minimal.

Directory Layout

images

The images directory contains a subdirectory for each HTTP server and transducer in the Garden. Each target gets its own Docker image. All programs are built from source when possible. So that we can easily build multiple versions of each target, all targets are paremetrized with a repository URL (APP_REPO), branch name (APP_BRANCH), and commit hash (APP_VERSION).

tools

The tools directory contains the scripts that are used to interact with the servers. Inside it, you'll find

  • diagnose_anomalies.py: A script for enumerating benign HTTP parsing quirks in the systems under test to be ignored during fuzzing,
  • repl.py: The primary user interface to the HTTP Garden,
  • update.py: A script that updates the commit hashes in docker-compose.yml,
  • ...and a few more scripts that aren't user-facing.

Targets

HTTP Servers

Name Runs locally? Coverage Collected?
aiohttp yes yes
apache_httpd yes yes
apache_tomcat yes no
cheroot yes yes
cpp_httplib yes no
dart_stdlib yes no
eclipse_grizzly yes no
eclipse_jetty yes no
fasthttp yes no
go_stdlib yes no
gunicorn yes yes
h2o yes yes
haproxy_fcgi yes no
hyper yes no
hypercorn yes yes
ktor yes no
libevent yes no
libmicrohttpd yes no
libsoup yes no
lighttpd yes yes
mongoose yes yes
netty yes no
nginx yes yes
node_stdlib yes no
openlitespeed yes no
openwrt_uhttpd yes yes
php_stdlib yes no
phusion_passenger yes no
protocol_http1 yes no
puma yes no
servicetalk yes no
tornado yes no
twisted yes no
undertow yes no
uvicorn yes yes
waitress yes yes
webrick yes no
yahns yes no
iis no no
openbsd_httpd no no

HTTP Transducers

Name Runs locally?
spring_cloud_gateway yes
apache_httpd_proxy yes
apache_traffic_server yes
go_stdlib_proxy yes
h2o_proxy yes
haproxy yes
haproxy_invalid yes
lighttpd_proxy yes
nginx_proxy yes
openlitespeed_proxy yes
pingora yes
pound yes
squid yes
varnish yes
yahns_proxy yes
akamai no
awselb_classic no
awselb_application no
cloudflare no
google_classic no
google_global no
iis_proxy no
openbsd_relayd no

Omissions

The following are explanations for a few notable omissions from the Garden: | Name | Rationale | | unicorn | Doesn't support connection reuse, and uses the same parser as yahns. | | SwiftNIO | Uses llhttp for HTTP parsing, which is already covered by node_stdlib. | | Bun | Uses picohttpparser for HTTP parsing, which is already covered by h2o. | | Deno | Uses hyper for HTTP parsing, which is already in the Garden. | | Daphne | Uses twisted for HTTP parsing, which is already in the Garden. | | pitchfork | Uses the same parser as yahns. | | nghttpx | Uses lhttp for HTTP parsing, which is already covered by node_stdlib. | | CherryPy | Uses cheroot for HTTP parsing, which is already in the Garden. | | libhttpserver | Uses libmicrohttpd for HTTP parsing, which is already in the Garden. | | Werkzeug | Uses the CPython stdlib for HTTP parsing, which is already in the Garden. | Cady | Uses the Go stdlib for HTTP parsing, which is already in the Garden. |

Results

See TROPHIES.md for a complete list of bugs that the Garden has found.