Kafka & Storm integration talk

This June I had the pleasure of attending the annual DORS/CLUC conference in Zagreb. It’s a convention for the Croatian Linux users group devoted to open systems in general. There was a bunch of interesting talks but my favorite of the day was definitely Mario Borna Mjertan, an elementary school student that’s been using Linux (as a user and as a developer) for 6 years already. Talking about a real interest in computers 🙂

Also, I had the pleasure of talking about integrating Kafka & Storm at my dev shop. The basic idea was to build a system that can process incoming data stream (in the range of about 5,000 events per second) and build end user dashboard data (trends, current submit rate etc.) in near real-time. In the talk I gave a short intro to the technology stack and basic concepts but also an overview of bottlenecks and issues we experienced along the way.

Below you can find the video of the talk:

And the related slides:

Storm overview & integration from Vanja Radovanović

Video can also be found here.