This post originated from an RSS feed registered with Java Buzz
by Brian McCallister.
Original Post: Silly Micro-Benchmarks
Feed Title: Waste of Time
Feed URL: http://kasparov.skife.org/blog/index.rss
Feed Description: A simple waste of time and weblog experiment
So, the recent rash of controversial benchmarks have inspired me to do some of my own microbenchmarking. I want to test web servers, and specifically *slightly* dynamic stuff, basically how long does it take to setup and teardown the request processing stuff without doing anything significant with the request (like talking to a database) as I want to isolate and benchmark the actual web part, not anything else. Luckily, there is a perfet test for this, Hello World! All tests were performed on the same machine, and I won't provide the code or configurations (as then they might be considered useful). I did tune each one to the best of my ability for running in the environment it is in (4 core machine, plenty of ram, etc). So, without further ado, results of a dynamic Hello World on a number of different webapp-ish servers, sorted by performance:
Server
Requests / Second
Notes
Apache HTTPD (Worker) via mod_hello_world
Requests per second: 18823.58 [#/sec] (mean)
This is basically a minimal httpd module which just prints hello world to the response
Apache HTTPD (Worker) via mod_wombat
Requests per second: 17856.76 [#/sec] (mean)
This uses a server-scoped mod_wombat handler with a default pool of fifty Lua virtual machines
Apache Tomcat 5.5.20
Requests per second: 17644.40 [#/sec] (mean)
This used a JSP and a few hundred thousand requests before the benchmark to let the JVM warm up.
Jetty 6.1.1
Requests per second: 12449.36 [#/sec] (mean)
This used a JSP and a few hundred thousand requests before the benchmark to let the JVM warm up.
Mongrel accessed directly
Requests per second: 2378.05 [#/sec] (mean)
This was done via an HttpHandler rather than Rails in order to cut down on overhead
Mongrel, four instances, proxied through Pen
Requests per second: 2109.91 [#/sec] (mean)
This configuration was basically a test to make sure that ruby's single-threaded nature wasn't the bottleneck. It seems the additional proxying was much more expensive than not using the extra four cores available.
TwistedWeb
Requests per second: 2089.55 [#/sec] (mean)
This is the current TwistedWeb, not TwiestedWeb 2.
Mongrel (four instances) proxied through LightTPD
n/a
This was a very unstable configuration. I was unable to run more than a thousand or so requests before lighttpd would start losing track of mongrel instances and start returning errors. LightTPD returns errors very quickly, however.
Now, to complicate matters further. Mongrel doesn't support HTTP keep alive, so I re-ran all the other benchmarks with keep alive disabled in the client and the numbers dropped about in half in every case, the three-way-handshake seems to matter a lot for micro-benchmarks.
Finally, this is a wholly unscientific set of benchmarks which is basically useless, so please don't read anything important into it. Any real system is almost certainly not bound by request handling in the application server :-)
Previous Topic
