Streaming data is a rapidly emerging category for new application development. The interest in apps that use streaming data is driven by the growth of machine data (think IoT and machine-to-machine communications) and the desire to improve online customer engagement through real-time personalization techniques. Definitions of streaming data applications vary, but they typically revolve around three essential capabilities: real-time data ingestion, real-time analysis, and automated decision making.
Stream processing solutions can also take many forms. Common architectures include tuple-at-a-time stream processing tools (aka complex event processing tools); batch-oriented stream processing solutions that trade latency for flexibility and robustness; and in-memory databases that can transact in real time. While these approaches can be used to solve many problems, they aren't all created equal in terms of developer effort, maintenance overhead, capabilities, or performance. In particular, it's important to look at the number of components required to provide the functionality you need. The amount of development, testing, and operations work to run a system is proportional to the number of components that must be glued together to solve a problem.
Previous Topic
