Thursday, February 18, 2010

Continuous Deployment Prereq #1 - Maker's Mark Nagiorb

At Knewton, we've used a build orb to continuous build success for the past 1.5 years.  We're currently working on Continuous Deployment for one of our subsystems.  As a prerequisite, we've setup nagios to monitor system state.  If you work at a startup, have customers and don't have nagios setup yet... start setting it up today.  It can easily monitor system state, mysql (replication monitoring is especially useful), and scripts for new probes are straightforward - for example, we're currently monitoring our zendesk support ticket queue, you can also synthetically test your web app.  Out of the box, nagios has email, and web-based monitoring, there are also scripts for sending IMs... The most useful of all of our custom extensions from @devondjones is the Maker's Mark Nagiorb.  Using an arduino, an LED array and a liquid etched, empty Maker's Mark bottle, our entire office now knows if all systems are fully functional, of if we're going to need another bottle of whiskey.

All Systems Go!
No warning, critical or unknown nagios alerts.
New ticket comes in from zendesk, or mysql replication behind.  Also, while a bit harder to show in static photos, the orb will blink with the number of open warning statuses.  So, if you have zendesk tickets and your mysql replication is behind, it will blink twice every minute.

Critical messages are similar to warnings, but will flash red instead of yellow. If there are several blinks, it's time to hit the full Maker's Mark bottle :)

If you're interested in having your own Nagiorb, you can get most of the way there by following Devon's build orb instructable.  I'll try to coax him into adding a sample nagios script in there as well.