This week, The Middleware Company has announced the TORPEDO O-R Benchmarking Initiative.
The "initiative" is a spec, and a reference implementation. It currently allows you to run a client against an EJB container, which uses different persistence tools on the back-end: Kodo JDO, Hibernate, WLS CMP, and TOPLink.
Benchmarks are always hard, nay impossible. TORPEDO is actually quite different. It doesn't try to measure performance based on the time that it takes to run a number of transactions. In fact, TORPEDO is designed in a way which means that you can grab it and run it on your laptop, and the results will be the same. The reason is, that the only metric CURRENTLY, is the SQL which comes out at the other end.
The assumption is that if the OR tool generates hundreds of SQL queries vs. ONE, then it isn't as efficient as it could be. This is an interesting approach. It isn't full proof, and more metrics will be added in the future I am sure. Some people want to measure the time that it takes to work out the queries, time more of the caching aspects, measure how many bytes are moving around etc. These are all good ideas, and TORPEDO has the ability to grow in the future to cover many different metrics, and together they will paint a great picture.
What would you like to see measured?
Previous Topic
