This article investigates and compares different design patterns of high performance TCP-based servers. In addition to existing approaches, it proposes a scalable single-codebase, multi-platform solution (with code examples) and describes its fine-tuning on different platforms. It also compares performance of Java, C# and C++ implementations of proposed and existing solutions.

http://www.artima.com/articles/io_design_patterns.html

What do you think of these design patterns?

try Twisted Matrix:
www.twistedmatrix.com

;)

what is the scale of the diagrams? If it is one connection/tick then that means the tests were done on not more then 30-40 connections. We made a few tests with a few thousands connections and we got very different results.

We have made tests with 50,100,200,500 and 1000 connections. So one tick approximately corresponds to 32 connections.

Try MINA for Java, too! :)

http://directory.apache.org/subprojects/network/

The emulated proactor design doesn't take into account that reads can be made concurrently on SMP systems. If handlers of I/O events run in multiple threads, it's more effective to let the handler threads call read/write on the ready descriptors.
An emulated proactor modified that way would also include thread pool machinery, which would make it monstrous and rife with side effects.

First of all, thanks for constructive response! Mikhail, you are absolutely right that on multi-CPU systems multiple reads/writes in parallel threads significantly improve performance. The detailed description of emulated Proactor was of the scope of this article. It the latest version of J5Proactor (http://www.terabit.com.au/J5Proactor-1.0.zip) code completion dispatcher works in handler thread. Actually, Proactor thread can be in one of 3 states: LEADER (checks/waits for readiness events), HANDLER (really performs I/O) and dispatches completions, FOLLOWER (waits for be leadership). When leader thread goes to HANDLER state, it passes leadership to the of the FOLLOWER threads and after that performs I/O and calls user handler. J5Proactor works much faster that on old JavaProactor-1.4 on multi-CPU boxes.
The diagrams show result of old Proactor (J5Proactor was under development at the time of preparation article).
I see nothing bad in design where event demultiplexor and thread pool are embedded in single entity. What is the better alternative? Thread-per-connection is easy, but not scalable. We have the old goal to process M connections in N threads, where
M >> N. I think optimal N is equal the number of CPUs; more threads don’t improve performance. So we still have to deal with thread pool. Thanks, again.

The actual problem at hand is that threads just eat too many resources from the OS to be really useful for this kind of system.

NT Fibers - which are a kind of thread, but without the scheduling - might be a nice approach to this.

For handling socket communication, the server creates a Fiber for every accepted connection. In the main loop, it uses a select call to see which sockets are ready for processing, and actives the corresponding fibers by attaching them to a thread. Control is returned when the socket returns EWOULDBLOCK.

Multi-processor systems can use multiple worker threads, and fibers can be assigned to any available thread.

This basically combines the simple approach of creating threads for each connection, and the low resource usage of a reactor. Each handler can just, for example, keep on reading from a socket, and process the results as if it were the only running object. This avoids the overhead of having to save current progress/status before returning control, which a system built on select() would be forced to do, as you described in your article. This administration is now done by the fiber.

If the simulated Proactor is more or less equivalen to a Reactor where does the 30% performance gain with simulated Proactor come from? The fact that the time spen by handlers to decide what the next I/O op should be is offset into worker thread?

Hi Todor, you are right, the emulated ideal Proactor could not be faster ideal Reactor, but it gives you the equal performance. The idea of article was: Proactor can give max performance that you get on current environment. If native AIO is faster - it will be based on it; if reactive approach is faster - Proactor will emulate AIO. In both cases user will maintain only single code base and deal with single pattern.

Why does it happen that Proactor tests demonstrated the better performance on Linux? Just because current Reactor implementation is not the best: single mutex for whole repository, not optimized work with notification pipe, slow mechanizm of obtaining leadership ...