My company has been considering using Spark on a client project that calls for lower latency than Hadoop can offer. The general evaluation has been about maturity, the flow of data in and out of memory, and the advantages of Spark versus more mature technologies. For this project, we didn't really consider Hadoop due to its nature.
Over the past week as Spark hit 1.0, a lot of lazy journalists have been describing Spark as a "Swiss army knife" of data processing and rattling off a bunch of press release marketing malarkey. Come on, people! If you're going to copy a press release into your column, at least reword it a bit! Nothing I read actually bothered to explain exactly what Spark is, why you would use it, or where it would fit into your Hadoop stack.
Previous Topic
