This post originated from an RSS feed registered with Ruby Buzz
by Daniel Berger.
Original Post: IO Completion Ports to the Rescue!
Feed Title: Testing 1,2,3...
Feed URL: http://djberg96.livejournal.com/data/rss
Feed Description: A blog on Ruby and other stuff.
In the ongoing quest to convert all of the Win32Utils libraries into pure Ruby (instead of C extensions), Park and I finished up the latest version of win32-changenotify. In short, this is a library for monitoring file and directory changes on MS Windows. It's deprecated for NTFS, but nevermind that! I'm on a mission!
One of the problems with both the old C extension and the early iterations of the pure Ruby version was that they just weren't....reliable. Oh, they would pick up change events, they just couldn't pick up all of them, or even most of them, most of the time. And sometimes they got confused by lots of simultaneous events. We seemed to be doing everything right, so I chalked it up to cruddy architecture.
That is, until Park (Heesob) came along with a version that ditched the WaitForSingleObject() approach and instead uses CreateIoCompletionPort() + GetQueuedCompletionStatus(). IO completion ports are one of the ways in which MS Windows does concurrency. In short, a completion port is a thread queue (of one or more threads) that monitors file handles. Kinda like select(), I suppose, except that the file handles have to be created with certain parameters (FILE_FLAG_OVERLAPPED) and there are only eight functions that can actually monitor completion status. The ReadDirectoryChangesW() function is one of these eight functions, and the one that win32-changenotify uses.
(Someone please correct me if I've summarized completion ports incorrectly)
The upshot of all this is that win32-changenotify has gone from being mediocre at monitoring file changes to very, very good. Not perfect, however. One of the downsides of not having native thread support is that we can't have more than one concurrent thread monitoring changes. This means that it's susceptible to a race condition between the call to GetQueuedCompletionStatus() and ReadDirectoryChangesW(). Park did some experiments using Ruby's green threads to fix this, but so far hasn't had any luck. He may come up with something yet, though.
In any case, the good news is that the faster your system is, the less likely you are to encounter gaps. Besides, even with this issue, it's still MUCH better than it was before. Hopefully, now that it's pure Ruby, more people will take a look at it and perhaps come up with ways to help. :)
Previous Topic
