Saiweb » Linux

Linux collection of handy scripts and one liners – Volume 2 (Warning: contains shortcuts)

Buzz — Mon, 26 Sep 2011 15:43:34 +0000

See if hosts are up using ping in range 60 -> 200

1
2
3
4
5

for i in {60..200}; do ping -c 1 -W 1 192.168.1.$i > /dev/null; ([[ $? == 0 ]] && echo "$i UP" || echo "$i DOWN"); done
1 UP
2 DOWN
3 UP
...

Note: for OSX use “ping -c 1 -t 1″

Chaining “UP” hosts for a quick (syn) port scan

1
2
3

for i in {60..200}; do ping -c 1 -W 1 192.168.1.$i > /dev/null; ({{ $? == 0 ]] && nc -v -n -z -w1 192.168.1.$i 20-22); done
(UNKNOWN) [192.168.1.1] 22 (ssh) open
(UNKNOWN) [192.168.1.3] 22 (ssh) open

Recover from a bad mysql password set (Update mysql.users set password=’Iforgotawherestatemenlulz’)

Assumes for every user there is an @localhost host, grabs the in memory password hash and resets

mysql -Bse 'Select distinct(user) from mysql.user;' | while read uname; do mysql -Bse "show grants for '$uname'@'localhost';" 2>&1 | grep IDENTIFIED | grep -v 'root' | grep -v 'ERROR' | sed 's|GRANT USAGE ON *.* TO ||g' | sed "s|@'localhost' IDENTIFIED BY PASSWORD||g" | awk '{print "Update user set Password="$2" where User="$1";"}' | mysql mysql; done

If you’ve run FLUSH PRIVILEGES; however you == b0ned.

Quick substitute and run

Command1:

1	ping -c 1 -t 1 192.168.1.1

Opps that’s OSX synatx

Command2:

1	^-t 1^-W 1

et voila corrected syntax.

Shortcuts

!! – Execute last command
!ping – Execute last ping command, can be used to !any command just be careful.
ctrl+r – reverse search, just start typing the cmd for it to search your history, hit tab to complete
ctrl+a – jump to beginning of line
ctrl+e – jump to end of the line

cURL FU

curl -I -L blahblah.tld – Run a HEAD and follow redirects (very handy for quicklooking @ bit.ly short URLS before hitting them in a browser).

python FU

python -m SimpleHTTPServer – serves the current `pwd` as a browseable directory (Very cool but VERY insecure)
python -m cProfile script.py – generate trace stats for a script execution (Very handy for finding excessive loops)

DNS Fu

Wikipedia over DNS:

host -t txt fu.wp.dg.cx

fu.wp.dg.cx descriptive text “Fu may refer to: Fu (Technology, especially computer related) (used as a suffix) – relating to a person – Possessing superior skills in an art\; relating to an artifact – representing an expression of high art. code-fu, Perl-fu, C-fu, etc, Fu (literature),” ” a Chinese genre of rhymed prose, Fu (kana), a symbol in Japanese syllabaries, Fu County, in Shaanxi, China, Fu Foundation… http://a.vu/w:Fu”

Useful on _some_ public wifi connections if you just want to look something up quick (dns is not always re-written).

Get all MX servers for a domain:

dig google.co.uk MX

; <<>> DiG 9.6.0-APPLE-P2 <<>> google.co.uk MX
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 64165
;; flags: qr rd ra; QUERY: 1, ANSWER: 4, AUTHORITY: 4, ADDITIONAL: 4

;; QUESTION SECTION:
;google.co.uk. IN MX

;; ANSWER SECTION:
google.co.uk. 10800 IN MX 10 google.com.s9a1.psmtp.com.
google.co.uk. 10800 IN MX 10 google.com.s9a2.psmtp.com.
google.co.uk. 10800 IN MX 10 google.com.s9b1.psmtp.com.
google.co.uk. 10800 IN MX 10 google.com.s9b2.psmtp.com.

;; AUTHORITY SECTION:
google.co.uk. 59925 IN NS ns2.google.com.
google.co.uk. 59925 IN NS ns3.google.com.
google.co.uk. 59925 IN NS ns4.google.com.
google.co.uk. 59925 IN NS ns1.google.com.

;; ADDITIONAL SECTION:
ns1.google.com. 158334 IN A 216.239.32.10
ns2.google.com. 158334 IN A 216.239.34.10
ns3.google.com. 158741 IN A 216.239.36.10
ns4.google.com. 158334 IN A 216.239.38.10

;; Query time: 68 msec
;; SERVER:
;; WHEN: Mon Sep 26 16:41:26 2011
;; MSG SIZE rcvd: 310

mySQL FU

in one line, take a database, in stream replace content and stream into another db.

mysqldump original_db | sed ‘s/content_or_regex_to_replace/content_or_backref_replacement/g’ | mysql destination_db
Tweet

Cloaking your web apps – The hooded Apache

Buzz — Mon, 25 Apr 2011 19:54:06 +0000

Go ahead and run

1	curl -I http://www.saiweb.co.uk

You will get

1
2
3
4
5
6
7
8

HTTP/1.1 200 OK
Date: Mon, 25 Apr 2011 19:33:29 GMT
Server: Apache
Vary: Accept-Encoding,Cookie
Cache-Control: max-age=3, must-revalidate
WP-Super-Cache: Served supercache file from PHP
Connection: close
Content-Type: text/html; charset=UTF-8

As an attacker looking to hit a web app, one of the first things you’re going to want to look into is what version of web server is running, in this case you can see this blog in fact runs apache … but there is not much else to go on here is there.

That’s intentional, and by manual configuration changes I have put in place, this is not the case of a default LAMP install, take for instance, this snippet from another website,

1 2	Server: Apache/2.2.16 (Debian) X-Powered-By: PHP/5.2.14

This already has given me a wealth of information to go on and begin prepping an attack, I now know the site is running php version 5.2.14 Apache version 2.2.16 and that the underlying OS is Debian.

See the dilemma? your default roll outs are just declaring their running versions to anyone willing to listen, so lets make it a little more stealthy.

First and foremost if you are using php, edit your php.ini and set the following:

1	expose_php = off

Now head into your httpd.conf and set the following

1	ServerTokens prod

and

1	ServerSignature off

With these 3 simple steps all the headers will now return is Server: Apache this is the first step to shielding your app, I’ll be covering further steps as time allows.

Linux collection of handy scripts and one liners – Volume 1.5 (Feedback edition)

Buzz — Fri, 15 Oct 2010 10:08:12 +0000

Following reader feedback please see below for an updated version of Volume 1

Ever wanted / needed HTTPD or another service to run with a raised thread priority?

Well you have a couple of options, add additional lines to the /etc/init.d script to change the nice level by adding additional lines on startup, or if you only need to do this on a temporary basis without restarting the service but need every thread to have a raised priority you can use a bash script

Much cleaner script here again thanks to Matthew Ife.

1 2	#!/bin/bash pgrep httpd \| while read pid; do renice -20 $pid; done

You can renice between -20 and +20, depending on your requirements you can use this script in a cron job to raise/lower priorities, change httpd for whatever service you want to change the thread priority for.

Ever needed to check files were being accessed / written to?

For this one you’re going to need the inotify-tools package, specifically the inotifywait binary.

1	inotifywait -m --timefmt "[%a %b %d %H:%M:%S %Y]" --format "%T [%e] %f" -r /folder/to/watch

An example usage is to ensure that caching is working correctly and that cache files are being used in place of processing PHP files, simply change “/folder/to/watch” to be your cache folder, and refresh a few pages.

All being well you’ll get an output similar to the following:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35

y-tools-3.14)
(root@132 BUZZ1) # /usr/local/bin/inotifywait -m --timefmt "[%a %b %d %H:%M:%S %Y]" --format "%T [%e] %f" -r /path/to/saiweb/wp-content/cache/supercache/*
Setting up watches. Beware: since -r was given, this may take a while!
Watches established.
[Thu Jul 15 20:59:37 2010] [OPEN] index.html
[Thu Jul 15 20:59:37 2010] [CLOSE_NOWRITE,CLOSE] index.html
[Thu Jul 15 21:00:08 2010] [OPEN,ISDIR]
[Thu Jul 15 21:00:08 2010] [OPEN,ISDIR] security
[Thu Jul 15 21:00:08 2010] [OPEN,ISDIR]
[Thu Jul 15 21:00:08 2010] [OPEN,ISDIR] vsftpd-chrooting-without-the-headache-allowing-shared-directories
[Thu Jul 15 21:00:08 2010] [OPEN,ISDIR]
[Thu Jul 15 21:00:08 2010] [CLOSE_NOWRITE,CLOSE,ISDIR] vsftpd-chrooting-without-the-headache-allowing-shared-directories
[Thu Jul 15 21:00:08 2010] [CLOSE_NOWRITE,CLOSE,ISDIR]
[Thu Jul 15 21:00:08 2010] [OPEN,ISDIR] the-zen-of-secured-shared-hosting-part-1
[Thu Jul 15 21:00:08 2010] [OPEN,ISDIR]
[Thu Jul 15 21:00:08 2010] [CLOSE_NOWRITE,CLOSE,ISDIR] the-zen-of-secured-shared-hosting-part-1
[Thu Jul 15 21:00:08 2010] [CLOSE_NOWRITE,CLOSE,ISDIR]
[Thu Jul 15 21:00:08 2010] [OPEN,ISDIR] php-security-considerations
[Thu Jul 15 21:00:08 2010] [OPEN,ISDIR]
[Thu Jul 15 21:00:08 2010] [CLOSE_NOWRITE,CLOSE,ISDIR] php-security-considerations
[Thu Jul 15 21:00:08 2010] [CLOSE_NOWRITE,CLOSE,ISDIR]
[Thu Jul 15 21:00:08 2010] [OPEN,ISDIR] antivirus-xp-2008-removal
[Thu Jul 15 21:00:08 2010] [OPEN,ISDIR]
[Thu Jul 15 21:00:08 2010] [CLOSE_NOWRITE,CLOSE,ISDIR] antivirus-xp-2008-removal
[Thu Jul 15 21:00:08 2010] [CLOSE_NOWRITE,CLOSE,ISDIR]
[Thu Jul 15 21:00:08 2010] [OPEN,ISDIR] suphplookupexception
[Thu Jul 15 21:00:08 2010] [OPEN,ISDIR]
[Thu Jul 15 21:00:08 2010] [CLOSE_NOWRITE,CLOSE,ISDIR] suphplookupexception
[Thu Jul 15 21:00:08 2010] [CLOSE_NOWRITE,CLOSE,ISDIR]
[Thu Jul 15 21:00:08 2010] [OPEN,ISDIR] honeypotting-for-viruses-statement-of-fees-200809
[Thu Jul 15 21:00:08 2010] [OPEN,ISDIR]
[Thu Jul 15 21:00:08 2010] [CLOSE_NOWRITE,CLOSE,ISDIR] honeypotting-for-viruses-statement-of-fees-200809
[Thu Jul 15 21:00:08 2010] [CLOSE_NOWRITE,CLOSE,ISDIR]
[Thu Jul 15 21:00:08 2010] [CLOSE_NOWRITE,CLOSE,ISDIR] security
[Thu Jul 15 21:00:08 2010] [CLOSE_NOWRITE,CLOSE,ISDIR]

Alternatively you can use the following approach contributed by Matthew Ife:

1	auditctl -w /some/path -p w

This will persist for the duration of your ssh session and relevant log entries will appear in /var/log/audit/audit.log, admittedly with far more useful information than inotifywait, and does not require you to install additional packages.

As can be seen the re-write rules are redirecting users to the cached files/folders, in the example above I have used my wp-supercache folder.

Ever needed to quickly get the memory usage of all threads for a service?

You have two options for this a single line

1	ps -Ao rsz,comm,pid \| grep

or a bash function you can place in your ~/.bashrc

1
2
3
4
5
6
7
8

function appmem(){
if [ -z "$1" ]; then
echo "appmem "
echo "i.e. appmem httpd";
else
ps -Ao rsz,comm,pid | grep $1
fi
}

You can then call this (after logging back in again to load the .bashrc up) using

appmem

replacing for instance with httpd will give you an output similar to the following:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

8032 httpd 6207
33080 httpd 13828
8552 httpd 14095
28952 httpd 14102
8540 httpd 14103
30848 httpd 16741
31296 httpd 16832
30452 httpd 18439
31044 httpd 19996
30968 httpd 23287
30356 httpd 23300
25636 httpd 24553
29712 httpd 24771
25588 httpd 24777
31632 httpd 24778
25608 httpd 24796
29716 httpd 24812
28152 httpd 24813
31684 httpd 31291

This shows memory in kilobytes, command, process id, you can see here I currently have 3mb/pid for each httpd process (due to my optimizations, I highly recommend you read parts 1-3)

Dump mysql data and compress on the fly

1	mysqldump -h -u -p \| bzip2 -c7 > /path/to/dump.sql.bz2

Self explanatory that one, pipes the output from mysqldump through bzip2 (which has better compression over gzip) and dumps it out to a file, if you _realy_ need a gziped file just replace bzip2 with gzip in the line above.

Ever needed a selection of passwords generated?

Using a slightly modified line originally provided by Matthew Ife,

1
2
3

function pwgen(){
dd if=/dev/urandom bs=2048 count=1 | tr -cd ‘a-zA-Z0-9+@\!\$’ | cut -b1-15
}

Plant this in your ~/.basrc for a callable function that will generate a selection of 10 secure passwords, handy when you’re fed up of 1337′ifying everything

example output:

1	)S9esjccl?MMiC1

If you want runtime variable length you could change to cut -1-$1 and then call pwgen 15 for example.

Check mySQL myISAM fragmentation

1
2

USE information_schema;
SELECT CONCAT(TABLE_SCHEMA,'.',TABLE_NAME) AS TABLE_NAME, ENGINE, (DATA_LENGTH/1024/1024) AS DATA_LENGTH, (INDEX_LENGTH/1024/1024) AS INDEX_LENGTH, ((DATA_LENGTH + INDEX_LENGTH)/1024/1204) AS TOTAL_LENGTH,TABLE_ROWS, UPDATE_TIME, ((INDEX_LENGTH/(DATA_LENGTH + INDEX_LENGTH))*100) AS INDEX_PER,((DATA_LENGTH/(DATA_LENGTH + INDEX_LENGTH))*100) AS DATA_PER, (DATA_FREE/DATA_LENGTH) AS FRAG_RATIO FROM TABLES WHERE ENGINE IS NOT NULL AND DATA_LENGTH >=(1024*1024) AND (DATA_FREE/DATA_LENGTH) >=0.02 ORDER BY FRAG_RATIO DESC;

Gives you a very quick overview of make up of your myISAM tables and their fragmentation (Data free vs data length).

Linux collection of handy scripts and one liners – Volume 1

Buzz — Thu, 15 Jul 2010 20:15:15 +0000

Ever wanted / needed HTTPD or another service to run with a raised thread priority?

1
2
3
4
5
6

#!/bin/bash
PIDS=`ps aux | grep httpd | grep -v 'grep' | awk '{print $2}'`;
for PID in ${PIDS[@]}
do
renice 20 -p $PID
done

Ever needed to check files were being accessed / written to?

For this one you’re going to need the inotify-tools package, specifically the inotifywait binary.

1	inotifywait -m --timefmt "[%a %b %d %H:%M:%S %Y]" --format "%T [%e] %f" -r /folder/to/watch

All being well you’ll get an output similar to the following:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35

As can be seen the re-write rules are redirecting users to the cached files/folders, in the example above I have used my wp-supercache folder.

Ever needed to quickly get the memory usage of all threads for a service?

You have two options for this a single line

1	ps -Ao rsz,comm,pid \| grep

or a bash function you can place in your ~/.bashrc

1
2
3
4
5
6
7
8

function appmem(){
if [ -z "$1" ]; then
echo "appmem "
echo "i.e. appmem httpd";
else
ps -Ao rsz,comm,pid | grep $1
fi
}

You can then call this (after logging back in again to load the .bashrc up) using

appmem

replacing for instance with httpd will give you an output similar to the following:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

This shows memory in kilobytes, command, process id, you can see here I currently have 3mb/pid for each httpd process (due to my optimizations, I highly recommend you read parts 1-3)

Dump mysql data and compress on the fly

1	mysqldump -h -u -p \| bzip2 -c7 > /path/to/dump.sql.bz2

Ever needed a selection of passwords generated?

For this one you can use the secpwgen

1
2
3

function pwgen(){
for (( i=0; i<=10; i++ )) do pwd=`secpwgen -Aadhs 10 2>&1 | grep ENTROPY | awk '{print $1}';`; echo "$i: $pwd"; done;
}

Plant this in your ~/.basrc for a callable function that will genrate a selection of 10 secure passwords, handy when you’re fed up of 1337′ifying everything

example output:

1
2
3
4
5
6
7
8
9
10
11

0: 4>&B.\2R+--
1: )`WREEGZP{
2: ^)3"=F==|?0
3: ?1/|;;GF-2
4: [..///_([=AZ
5: }^%RC~U8//L
6: \//VNTQ[)->
7: @HE5@3)A%?
8: )|1C[BSIT*
9: C[//X^W<$G1
10: EOQ#Y%NI>-

Modify the “-Aadhs” args to your taste.

This concludes Volume 1 and a very long post, please contribute your one liners / helper scripts via the comments.

Cheers

buzzTweet

/bin/sh: bad interpreter

Buzz — Mon, 01 Feb 2010 11:41:58 +0000

For security newer distros of RHEL and their derivatives an mounting /tmp with the noexec option.

Now if you have ever had to clean up a compromised web app you can see why this makes a lot of sense, and if not here’s a quick example.

Yours/Clients web app becomes compromised, running kernel has a buffer overflow that can lead to privilege escalation, attack writes out their code and compiles in /tmp, then runs said app from /tmp creating a pseudo root level shell, aka you’ve just been root kitted.

However there are legitimate reasons for using /tmp to compile, well I say legitimate, what I in fact mean is things like pecl, which you use to install extensions like APC require this …

workaround:

1	export TMPDIR='/a/paTh/your/user/can/write/to'

Failing that:

service httpd stop

DO NOT ALLOW ANY WEBAPP ACCESS WHILE NOEXEC IS IN USE!

1
2
3

mount -o,remount,rw,exec /tmp
pecl install apc
mount -o,remount,rw,noexec /tmp

DO NOT REMOVE THE NOEXEC OPTION IN /ETC/FSTAB PERMANENTLY YOU WILL REGRET DOING SOTweet

Linux – Generating file manifests and then checking them – Improved

Buzz — Tue, 29 Sep 2009 14:14:06 +0000

Following on from Linux – Generating file manifests and then checking them I was always getting the same questions …

How long left on the manifest Buzz ?
How long left on the verification Buzz ?

And I HATE having to turn around an say … I don’t know …

The problem with the usual command line method is that it give no indication of progress, and by extension no indication that it was infact running and not ‘hung’ …

As such I have now added the ‘manifest’ command set to the Sysadmin toolset

The manifest command take two data types, the first is a folder path from which to build the file manifest from, the manifest itself is also compatible with the “md5sum –check” function.

The second is the path to the manifest itself, in this case the manifest command will verify each file against it’s entry in the manifest:

At each point the command give you an indication of it’s current status, however this does come at a small cost, the script has no concept of the size of you console and as such will always render out the same number of character meaning if you console is not wide enough it will not render correctly, in the videos I have the console on a high resolution monitor as can be seen each video itself is 900 pixels wide.

This process is CPU intensive (20-45% on one core of a intel core2duo 2.8GHZ) and uses around 140KB of memory.Tweet

Linux – The Sysadmin Script – Part 4

Buzz — Fri, 14 Nov 2008 09:31:23 +0000

In part 4, I am going to cover more of an improvement than anything else to part 3

Part 3 itself is not incorrect, it correctly takes a memory footprint for each process running, the same as VIRT in top …

However in processes such as APACHE the VIRT memory is the size of all shared libraries, as correctly shown by pmap …

So what does this mean realy?

The memory usage is infact the following VIRT + RSS, where RSS is the resident set size, the RSS is a representation of the memory in use by the PID, and VIRT is shared between the child processes.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

[buzz@buzz_srv ~]# ps aux | grep httpd | grep -v 'grep'
root 16378 0.0 0.1 148640 3024 ? Ss Nov13 0:00 /usr/sbin/httpd
apache 20088 0.0 0.1 148640 3304 ? S Nov13 0:00 /usr/sbin/httpd
apache 20101 0.0 0.1 148640 3304 ? S Nov13 0:00 /usr/sbin/httpd
apache 20756 0.0 0.1 148640 3312 ? S Nov13 0:00 /usr/sbin/httpd
apache 20759 0.0 0.1 148640 3300 ? S Nov13 0:00 /usr/sbin/httpd
apache 20790 0.0 0.1 148640 3284 ? S Nov13 0:00 /usr/sbin/httpd
apache 20792 0.0 0.1 148640 3312 ? S Nov13 0:00 /usr/sbin/httpd
apache 20798 0.0 0.1 148640 3308 ? S Nov13 0:00 /usr/sbin/httpd
apache 20804 0.0 0.1 148640 3308 ? S Nov13 0:00 /usr/sbin/httpd
apache 20886 0.0 0.1 148640 3304 ? S Nov13 0:00 /usr/sbin/httpd
apache 20906 0.0 0.1 148640 3300 ? S Nov13 0:00 /usr/sbin/httpd
apache 20907 0.0 0.1 148640 3308 ? S Nov13 0:00 /usr/sbin/httpd
apache 20912 0.0 0.1 148640 3304 ? S Nov13 0:00 /usr/sbin/httpd
apache 20915 0.0 0.1 148640 3312 ? S Nov13 0:00 /usr/sbin/httpd
apache 20959 0.0 0.1 148640 3304 ? S Nov13 0:00 /usr/sbin/httpd
apache 20969 0.0 0.1 148640 3300 ? S Nov13 0:00 /usr/sbin/httpd
apache 20994 0.0 0.1 148640 3320 ? S Nov13 0:00 /usr/sbin/httpd
apache 20995 0.0 0.1 148640 3288 ? S Nov13 0:00 /usr/sbin/httpd
apache 20996 0.0 0.1 148640 3320 ? S Nov13 0:00 /usr/sbin/httpd
apache 20997 0.0 0.1 148640 3320 ? S Nov13 0:00 /usr/sbin/httpd
apache 20999 0.0 0.1 148640 3296 ? S Nov13 0:00 /usr/sbin/httpd

As can be seen above the ‘VIRT’ does not change between the child processes, where as the RSS does dependant on what the thread is doing at that time.

So below is an improved appmem function to allow for this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

function appmem {
if [ -z "$1" ]; then
echo "Usage: sysadmin appmem app_name i.e. (sysadmin appmem apache)";
else
RRES=(`ps aux | grep "$1" | grep -v 'grep' | grep -v "$0" | awk '{print $6}'`);
VRES=(`ps aux | grep "$1" | grep -v 'grep' | grep -v "$0" | awk '{print $5}'`);
COUNT=0;
VMEM=0;
RMEM=0;
for RSS in ${RRES[@]}
do
RMEM=$(($RSS+$RMEM));
done;
for VIRT in ${VRES[@]}
do
VMEM=$(($VIRT+$VMEM));
COUNT=$(($COUNT+1));
done;
VMEM=$(($VMEM/$COUNT));
VMEM=$(($VMEM/1024));
RMEM=$(($RMEM/1024));
echo -e "$YELLOW ----- MEMORY USAGE REPORT FOR '$1' ----- $CLEAR";
echo "PID Count: $COUNT";
echo "Shared Mem usage: $VMEM MB";
echo "Total Resident Set Size: $RMEM MB";
echo "Mem/PID: $(($RMEM/$COUNT)) MB";
fi
}

Example output:

1
2
3
4
5

----- MEMORY USAGE REPORT FOR 'httpd' -----
PID Count: 41
Shared Mem usage: 140 MB
Total Resident Set Site: 95 MB
Mem/PID: 2 MB

Linux – The Sysadmin script – Part 3

Buzz — Wed, 12 Nov 2008 12:25:22 +0000

PART 3 IS INACCURATE, THE BELOW SCRIPT IS FOR REFERENCE ONLY, IT HAS BEEN REPLACED IN PART 4

In part 3, I am going to cover a bash function that will allow you to profile the memory usage of any application by name.

By adding the function below into your script you can execute a command such as: sysadmin appmem apache

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

function appmem {
if [ -z "$1" ]; then
echo "Usage: sysadmin appmem app_name i.e. (sysadmin appmem apache)";
else
if [ -x '/usr/bin/pmap' ]; then
APID=(`ps aux | grep "$1" | grep -v 'grep' | grep -v "$0" | awk '{print $2}'`);
COUNT=0;
AMEM=0
for PID in ${APID[@]}
do
TMP=$((`pmap -x $PID | grep "total" | awk '{print $3}'`));
AMEM=$(($AMEM+$TMP));
COUNT=$(($COUNT+1));
done
AMEM=$(($AMEM/1024));
echo -e "$YELLOW ----- MEMORY USAGE REPORT FOR '$1' ----- $CLEAR";
echo "PID Count: $COUNT";
echo "Mem usage: $AMEM MB";
echo "Mem/PID: $(($AMEM/$COUNT)) MB";
echo -e "$RED"
echo -e "For more information run: pmap -x $PID $CLEAR";
else
echo 'Could not execute /usr/bin/pmap ... aborting';
exit;
fi
fi
}

Sample output:

1
2
3
4
5
6

----- MEMORY USAGE REPORT FOR 'apache' -----
PID Count: 6
Mem usage: 1134 MB
Mem/PID: 189 MB

For more information run: pmap -x 123456

You can of course replace ‘apache’ with the application or daemon name you want to profile the memory usage of.

This script does require that pmap is installed, if the script can not find it, it will abort.

As always any problems, post a comment.

UPDATE: Apparently I need to point out that if you haven’t read PART 2! then the colored output will not work … That’s why this entry is titled part 3, it does assume a degree of competence on your part in realizing part’s 1 and 2 may just be required reading …

NOTE: The above provides a complete memory footprint of the indvidual PID, the same as VIRT in top.

VIRT — Virtual Image (kb)
* The total amount of virtual memory used by the task. It includes all code, data and shared libraries plus pages that have been swapped out.
* VIRT = SWAP + RES Tweet

/usr/bin/ld: skipping incompatible /usr/lib/libcom_err.so when searching for -lcom_err

Buzz — Thu, 16 Oct 2008 10:02:28 +0000

/usr/bin/ld: skipping incompatible /usr/lib/libcom_err.so when searching for -lcom_err

his one has been bugging me for a couple of hours now, when trying to compile PHP on a 64bit OS …

Simple put it’s a missing symlink, and the config script is trying to “failover” to the version is can find which is 32 bit …

ln -sf /lib64/libcom_err.so.2 /lib64/libcom_err.so

Et voila fixed!

Linux – The Sysadmin script – Part 2

Buzz — Tue, 14 Oct 2008 08:40:55 +0000

Part 2 has finally arrived …. don’t all cheer at once now …

In part two I will cover how to run an IP range scan using bash script, and if the host can be pinged retrieve the MAC address of the connected host.

Now bare in mind this script was written to run from a MAC running OSX Leopard.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

#!/bin/bash
#colours
function colours {
CLEAR='\e[00m';
GREEN='\e[0;32m';
RED='\e[0;31m';
YELLOW='\e[1;33m';
}
#ipscan
function ipscan {
IPS_START=1;
IPS_END=254;
IPS_RANGE=192.168.1.
echo "Now running IPSCAN $IPS_RANGE$IPS_START - $IPS_RANGE$IPS_END"
for ((i=$IPS_START;i<=$IPS_END;i+=1)); do
RESULT=`ping -c 1 -t 1 $IPS_RANGE$i | grep "bytes from"`;
if [ -z "$RESULT" ]; then
echo -e "$IPS_RANGE$i:$RED DEAD $CLEAR";
# If you comment out the above to report just the alive hosts, bash gets a bit funny about not processing anything here, so uncomment the below to keep it happy
#holder=$i;
else
MAC=`arp $IPS_RANGE$i | awk '{ print $4 }';`;
echo -e "$IPS_RANGE$i:$GREEN ALIVE $CLEAR ($MAC)";
fi
done
}
colours;
$1 $2

To make this work on your Linux distro replace -t in the ping command with -W and check the awk entry for the arp output, a display of (no) means that no ARP entries could be found for the host, and change the IP range to that of your network.

That’s it for this part, dump this is a file and chmod +x as useual and run with ./script.sh ipscan.Tweet