This post originated from an RSS feed registered with Java Buzz by News Manager. Original Post: Big data messaging with Kafka, Part 1 Feed Title: JavaWorld Feed URL: http://www.javaworld.com/index.rss Feed Description: JavaWorld.com: Fueling Innovation Latest Java Buzz Posts Latest Java Buzz Posts by News Manager Latest Posts From JavaWorld

When the big data movement started it was mostly focused on batch processing. Distributed data storage and querying tools like MapReduce, Hive, and Pig were all designed to process data in batches rather than continuously. Businesses would run multiple jobs every night to extract data from a database, then analyze, transform, and eventually store the data. More recently enterprises have discovered the power of analyzing and processing data and events as they happen, not just once every few hours. Most traditional messaging systems don't scale up to handle big data in realtime, however. So engineers at LinkedIn built and open-sourced Kafka: a distributed messaging framework that meets the demands of big data by scaling on commodity hardware.

To read this article in full or to leave a comment, please click here