Saiweb

Prompted by the following remarks today …

Kerm: “;) there is always an abbreviation in the CLI as all sysadmins are lazy feckers”

Kerm: “Someone might think you actually do work occasionally, god forbid!”

Sysadmins are NOT inherently lazy, we just know how to save time, and are quite adept at doing so …ok?

You cheeky sods!

So let me clear up one instance in which I take a lot of information, and make it quickly and easily accessible using a “Lazy feckers” abbreviation …

Be warned this is a very jaded write up, read on at your own peril.

Right then, onto the point of this post, the sysadmin script part 1, this is going to cover how to check how many connections to a specific port you have on your server.

Trust me this becomes very useful when you have exhausted all other options when trying to figure out why your web server is running like a dog with no legs …

After running the above on your SSH session you will see lines, and lines … and yet more lines of network connection information, especially if you just run this on a busy server.

Example (colours added):

tcp 0 0 ***.***.***.***:25 ***.***.***.***:32794 ESTABLISHED

Key:

PROTOCOL Tx Rx LOCALHOST:PORT FOREIGN_HOST:PORT CONNECTION STATE

From this information it’s pretty easy to spot this is an inbound SMTP connection.

(If you can’t see why, don’t worry it’s ok maybe it’s genetic)

Now this may be handy, but other than taking all this information and dumping it into a spreadsheet (god knows you love those spreadsheets !!! ), how are you going to figure out how many connections are occurring from that external host?

How infact are you going to be able to easily see how many total connections to that port you have ?!?!

Bash script, now for some history, Bash is the Bourne Again Shell, or as I like to think of it, it is the verb for what I will do to your head if you ask me what BASH / SSH / Shell is again …

Now create a directory:

Note the prefixing dot, this will create a “hidden” directory in your home directory (~), the reason for this is so you don’t have system admin script sat in your home directoy, as if you are like me, all sorts of crap moves in an out of that directory on a daily basis, and the last thing you want to do is to have to rummage through backups trying to find “that script you wrote to diagnose connection problems a year ago“.

The point is these scripts will become part of your workflow, once written they will rarely need updating, and should never be called directly, (I mean we’re lazy right? WTH do we want to be typing the full script path for? … oh yeh it saves time!).

In this case:

You can of course call your script whatever you want, and use any text editor you want, if you don’t like / know vi …

Ok so the above code is provided with two functions usage and portcon.

MAKE SURE YOU REPLACE “ABC.DEF.HIJ.KLM” WITH YOUR LOCAL IP ADDRESS

CHMOD this file to allow execution.

Now edit your bashrc file.

And add the following:

alias buzz=’~/.sysadmin/buzz.sh’

Now exit (logout) your SSH session and log back in (or SU root > SU your_user for testing).

Now run the portcon check …

(Yes before you ask ***.***.***.*** does display the correct IP address, I have purposely removed them for security).

So, I have taken something that would of resulted in netstat output > spreadsheet to formulas > at a estimate 30mins a time analysis to something that now takes less than 5 seconds to type, and get the relevant output, for roughly the same initial effort (30 mins scripting time).

You could argue you can keep a spreadsheet pre-setup with the right formulas / pivot tables and just dump the data each time, well yes you could but that’s no where near as quick as this …

And no trying to convince me it is as quick and better than the script above, for

1. You have to wait for excel to open the spreadsheet
2. You have to copy paste the data
3. You have to wait for excel to process the formulas

If you have a machine that can do that in time equal to or less than the time it takes the script above to output the data, the only thing I have to say is, stop spending such a budget on desktops and get a better server.

Final Thoughts:

This write up is in jest, and is intended to be read as such, the code and methods provided above are factual. etc …Tweet

So while reading Hackaday today I came across a “render farm” project which has about the same power usage as a desktop (400w) the difference is this one has 24cpu’s and 48gb of ram.

Tweet

a very basic video tutorial covering how to compile with debug information, load your app into gdb, set breakpoints and step through it.

No where near as much details as I wanted, but I never seem to get time to finish the full version, this will help in the interim.

NOTE: you will need XviD codec or use VLC.

If you re-distribute the video please give a link back (don’t force me to start water marking).

So I find myself needing to tweak my Nagios installation a little bit, in this case I found the need for “out of hours” SMS alerts.

Nagios doesn’t cater for this natively, rather it does however allow you to create your own custom commands, this allows you to specify a script to be executed.

Now I am going to assume you are already quite familiar with Nagios , so here is the command definition from my installation.

# ‘alert-by-sms’ command definition
define command{
command_name alert-by-sms
command_line /etc/nagios/alert-by-sms.php “** $NOTIFICATIONTYPE$ alert – $HOSTALIAS$/$SERVICEDESC$ is $SERVICESTATE$ **”
}

As you can see all this command definition realy does is execute a php script, bear in mind that

“/path/to/php /path/to/script ”

as the command_line does not seem to work, so just add “#!/path/to/php -q” to the top of the php script (before the opening <?PHP tag). and CHMOD +X the file.

The php script used here takes $argv[1] and passes it into a function specific to the SMS api I use, the phone number and API definitions are hard coded ito the script.
You don’t really need me to upload my script, and if you do then you shouldn’t be attempting this …

Basically Nagios will execute the script, as defined at command_line, the script can do anything you choose.

Now to implement the command so it is actually used, I am pretty sure this entry in “timeperiods.cfg” is the default but just incase here it is.

# ‘nonworkhours’ timeperiod definition
define timeperiod{
timeperiod_name nonworkhours
alias Non-Work Hours
sunday 00:00-24:00
monday 00:00-09:00,17:00-24:00
tuesday 00:00-09:00,17:00-24:00
wednesday 00:00-09:00,17:00-24:00
thursday 00:00-09:00,17:00-24:00
friday 00:00-09:00,17:00-24:00
saturday 00:00-24:00
}

This is what I use for the “out of hours” definition, now to implement the SMS alerting, for this I have simply created a new contact definition in “contacts.cfg”, granted this means there are now two contact definitions for myself.

define contact{
contact_name out_of_hours
alias Out Of Hours Mobile

service_notification_period nonworkhours
host_notification_period nonworkhours
service_notification_options c,u,r,f
host_notification_options d,u,r
service_notification_commands alert-by-sms
host_notification_commands alert-by-sms
email HIDDEN EMAIL

}

This can be further customized depending on your setup, in this case the contact is me and I want to receive alerts for all servers & services, so I just add the contact “out_of_hours” into the admins contact group.

define contactgroup{
contactgroup_name admins
alias Nagios Administrators
members nagios-admin,out_of_hours
}

So there you have it, you now have the ground work to potentially make Nagios fire you alerts anyway you like, you could go as far as having it call you via attached modem, if you _realy_ want, but when you want your servers talking to you via phone call is the day you need to switch to decaff, and head out to the pub once in a while.

Now just “nagios -v /path/to/nagios.cfg” to do a quick sanity check and make sure there are no errors (if you have any go back and fix them and run nagios -v again!), if all is ok /etc/init.d/nagios restart (or equivalent for your distribution).

As always if you run into problems drop me a comment Tweet