This post originated from an RSS feed registered with Ruby Buzz by Eigen Class. Original Post: A simple full-text search engine in 200 lines of Ruby Feed Title: Eigenclass Feed URL: http://feeds.feedburner.com/eigenclass Feed Description: Ruby stuff --- trying to stay away from triviality. Latest Ruby Buzz Posts Latest Ruby Buzz Posts by Eigen Class Latest Posts From Eigenclass

I'd been planning to add full-text search capabilities to FastRI from the beginning, and in Ruby-land "full-text" means Ferret. But I wanted to keep dependencies to a minimum, as FastRI could someday aspire to replace parts of the agonizing ri in the stdlib. There had to be a middle ground between risearch's simplicity and slowness and Ferret's nuclear-powered nutcracker.

I decided to write a straightforward full-text engine using suffix arrays, and the very first (utterly na��ve) implementation was already fast enough for FastRI: taking ~10 seconds to index some 40 gems and the stdlib documentation, I was getting sub-millisecond query times. In a latter test, I indexed 20% of Linux' sources (why only 20%? because indexing was done in memory), which took half a minute, and queries were in the millisecond range. Not bad at all for a couple hundred lines of easy Ruby code.