Nagios IRC Notifications

30 Jun 2012

Lately (as I earlier pointed out on my blog) I’ve been working on improving GNOME’s infrastructure monitoring services. After configuring XMPP it was time to find out a good way for sending out relevant notifications to our IRC channel hosted on GIMPNET. I achieved that with a nice combo: supybot + supybot-notify, all that mixed up with a few grains of Nagios command definitions.

But here we go with a little step-by-step guide:

Requirements

  1. Install supybot and configure a new installation:
apt-get install supybot or yum install supybot
mkdir /home/$user/nagbot && cd /home/$user/nagbot
supybot-wizard (follow the directions to get the bot initially configured)
  1. Install and load the supybot-notify plugin by doing:
git clone git://git.fedorahosted.org/supybot-notify.git && cd supybot-notify
mkdir -p /home/$user/nagbot/plugins/notify && cp -r * /home/$user/nagbot/plugins/notify

Finally, load the plugin. (this will require you to authenticate to the bot)

Nagios configuration

Add the relevant command definitions to the commands.cfg file:

# 'notify-by-ircbot' command definition
define command{
    command_name    notify-by-ircbot
    command_line    /usr/bin/printf "%b" "#channel $NOTIFICATIONTYPE$ - $HOSTALIAS$/$SERVICEDESC$ is $SERVICESTATE$: $SERVICEOUTPUT$ ($$(hostname -s))" | nc -w 1 localhost 5050
    }

# 'host-notify-by-ircbot' command definition
define command{
	command_name	host-notify-by-ircbot
	command_line	/usr/bin/printf "%b" "#channel $NOTIFICATIONTYPE$ - $HOSTALIAS$ is $HOSTSTATE$: $HOSTOUTPUT$ ($$(hostname -s))" | nc -w 1 localhost 5050
	}

* adjust the Netcat’s host and port to your needs, in my case Supybot and Nagios were running on the same host. In the case of a Supybot running on a different host than Nagios, tweak Iptables to allow the desired port:

-A INPUT -m state --state NEW,ESTABLISHED -m tcp -p tcp --dport 5050 -j ACCEPT

Add a new entry on the contacts.cfg file:

define contact{
 	contact_name    nagbot
   	use             generic-contact        
        alias		Nagios IRC Bot
        email           example@example.com
        service_notification_commands   notify-by-ircbot
 	host_notification_commands      host-notify-by-ircbot
}

Reload Nagios:

sudo /etc/init.d/nagios3 reload

And finally, enjoy the result:

PROBLEM - $hostalias/load average is CRITICAL: CRITICAL - load average: 30.45, 16.24, 7.16 (nagioshost)
 RECOVERY - $hostalias/load average is OK: OK - load average: 0.06, 0.60, 3.65 (nagioshost)