We've been using Lucene, the popular Open Source search engine, on our recent projects. We're pretty happy overall, but had some problems as well. We've inherited and a pretty interesting architecture built around slicing and dicing XML, putting the results into a Lucene index and then deploying the index to our various application servers. We don't use our index for free-text searches; we use it for "parametric" searches. Parametric searches are equivalent to "where" clauses in SQL. We have lots of use-case specific variations, such as "type:blog subcategory:${blog-name}" or "in_archive:yes category:news." We also have a homegrown filter that automatically tacks on standard parameters such as date-range filtering to make sure that results are "live".