Qianqian Zhong, Xu Zhang and Zuofei Wang of Airbnb give a talk at a meetup in San Francisco where they discuss how they’re utilizing Kafka at Airbnb. First, they start with providing some of their Kafka/Airbnb metrics, scale and architecture until they transition into two huge use cases: Spinal Tap and Mussel. The three discuss these individual use cases with incredible detail, from event processing, evolution, scalability, fault tolerance all the way to sharding, real time mutation, partition rebalancing and so much more. This talk is packed with useful information on how they put Kafka into practice at Airbnb.
0:00 Start of presentation/agenda
0:55 Kafka at Airbnb: metrics, scale and architecture
2:45 Use Cases
2:57-18:22 Spinal Tap: requirements, workflow, event processing, table schema, evolution, destination, scalability, fault tolerance, etc.
18:23 Mussel: requirements, architecture, sharding for bulk load, real time mutation, query data, partition rebalancing
Community Slack: https://cnfl.io/slack
Confluent, founded by the creators of Apache Kafka®, enables organizations to harness business value of live data. The Confluent Platform manages the barrage of stream data and makes it available throughout an organization. It provides various industries, from retail, logistics and manufacturing, to financial services and online social networking, a scalable, unified, real-time data pipeline that enables applications ranging from large volume data integration to big data analysis with Hadoop to real-time stream processing. To learn more, please visit https://confluent.io
#confluent #apachekafka #kafka