Mesh/Sensu Page

See Also: Monitoring

notes about the install and config of a sudomesh monitoring server - ChrisJ project, chris@freeranger.com

Read all about Sensu here: http://sensuapp.org/docs/0.12/guide

The main reason for doing this is to be able to monitor the mesh nodes for performance, up/down status, etc. The concept is to be able to use the light-weight mini-snmpd daemon on the mesh nodes and use sensu to perform checks and gather performance statistics. The statistics will be used to track performance for ongoing improvements in mesh node software.

As a start, we will track some internal machines, get comfortable with that, then move on to the nodes.

Basics

Installed on small frame Dell PC, service tag: 2FDSGC1, green tape with info on the front.

• Debian
• 192.168.42.65
• monitor.local - (not accessible, perhaps dns is not correctly defined)
• user sudoroom:sudomesh
• The sensu web page can be accessed internally at:
  • http://192.168.42.65:8080/# (as of 2014.07.25 no services defined)
  • user: admin
  • pw: secret
• github repo: not yet. soon.

Restart Services

You can start, stop, restart the basic services like this:

/etc/init.d/sensu-api restart
/etc/init.d/sensu-server restart
/etc/init.d/sensu-client restart
/etc/init.d/sensu-dashboard restart

/etc/init.d/rabbitmq-server restart

Things to do

1. install sensu server
   1. set up SSL certificates
   2. install/config RabbitMQ
   3. install Redis
2. config monitoring for internal PCs and/or other instances - see issues, below
3. configure a set of checks to perform
   1. set up monitoring for switches - maybe?
   2. install mini-snmpd (http://wiki.openwrt.org/doc/howto/snmp.server) on mesh nodes and configure a server side check to access nodes via snmp directly. Should be no other node/client side requirement.
4. install sensu advanced admin page (https://github.com/sensu/sensu-admin)
5. install/build handlers for email, sms notifications
6. hook up performance data to charts
7. configure RabbitMQ admin web

Sensu Config Files

Look for sensu config files in:

/etc/sensu/conf.d/

api.json

{
  "api": {
    "host": "localhost",
    "port": 4567,
    "user": "admin",
    "password": "secret"
  }
}

dashboard.json

{
  "dashboard": {
    "port": 8080,
    "user": "admin",
    "password": "secret"
  }
}

rabbitmq.json

{
  "rabbitmq": {
    "ssl": {
      "cert_chain_file": "/etc/sensu/ssl/cert.pem",
      "private_key_file": "/etc/sensu/ssl/key.pem"
    },
    "host": "monitor",
    "port": 5671,
    "vhost": "/sensu",
    "user": "sensu",
    "password": "secret"
  }
}

client.json

{
  "client": {
    "name": "monitor",
    "address": "192.168.42.65",
    "subscriptions": [ "all","webservers" ]
  }
}

server.json

oops...

check_cron.json

{
  "checks": {
    "cron_check": {
      "handlers": ["default"],
      "command": "/etc/sensu/plugins/check-procs.rb -p crond -C 1 ",
      "interval": 60,
      "subscribers": [ "webservers" ]
    }
  }
}

Tricks, Things to know

• EMBEDDED_RUBY=true in /etc/default/sensu - this makes ruby available for the omnibus install of sensu. Always do this on fresh installs of sensu omnibus.

Issues

Installing Sensu on dev exit node

Trying to get the sensu client install on the dev exit node, exit1.sudomesh.org (192.157.221.200). Could not successfully run these commands as sudo:

notes from this page: http://sensuapp.org/docs/0.12/packages

wget -q http://repos.sensuapp.org/apt/pubkey.gpg -O- | sudo apt-key add - 
echo "deb     http://repos.sensuapp.org/apt sensu main" > /etc/apt/sources.list.d/sensu.list
apt-get update
apt-get install sensu