Real-Time Streams at Scale with Kafka

Ewen Cheslack-Postava ofConfluent Presented at 2015 Couchbase Connect

Since being open sourced, Apache Kafka has been widely adopted by organizations ranging from web companies like Uber, Netflix, and LinkedIn to more traditional enterprises like Cerner, Goldman Sachs, and Cisco. These companies use Kafka in a variety of ways: 1) as a pipeline for collecting high-volume log data to load into Hadoop, 2) as a means of collecting operational metrics to feed monitoring/alerting applications, 3) for low latency messaging use cases, and 4) to power near real time stream processing. In this talk, Ewen and David will 1) discuss how companies are using Apache Kafka, 2) explore how its unique architecture enables it to be used for both real time processing and as a bus for feeding batch systems like Hadoop, and 3) describe where it fits in the Big Data ecosystem.