birding: streamparse/kafka demo

birding is an open source project to produce a stream of recent twitter activity based on a sequence of search terms, using only twitter’s public APIs. It serves as both a standalone project and a demo of distributed real-time computing with Python using Storm/streamparse and Kafka/pykafka.

Problem statement & topology describes the problem and how it fits into a topology. Downloading and running birding describes how to interact with birding for development, demo, or light usage. A tour of birding’s implementation provides a light introduction to internals. Using birding in production discusses how birding is packaged for production use in an existing streamparse project. Configuring birding discusses various options for birding behavior when running locally or in production.

• Problem statement & topology
  • Problem Statement
  • Specifics
  • Observations
  • Topology
  • Other Goals
• Downloading and running birding
• A tour of birding‘s implementation
  • Python Twitter Client
  • Twitter API
  • Search Manager
  • Storm Bolts
  • Storm Spouts
  • Storm Topology
• Using birding in production
• Configuring birding
• Searching Gnip
• API

To discuss this project, join the streamparse user group.

Documentation Index