Saiweb » Buzz

RedHat mock your SCM

Buzz — Fri, 03 Feb 2012 16:33:19 +0000

The mock tool can be a wonderful thing, allowing you to produce rpm packages for any rpm based system (assuming your have the written .cfg for it).

What I did find a little lacking on the documentation side was the SCM integration (read: Source Control Management), git/svn etc …

In short so long as your rpm spec file is in your SCM (and it should be), moc will build your rpm from your sources in scm, which can be used for.

1. bleeding edge builds for testing
2. builds from “stable tags”

Yes yes yes … obvious I know …

So with no futher ado here is the syntax:

1	mock -r your_target --scm-enable --scm-option method=git --scm-option package=git_project --scm-option git_get='git clone git@git_ip_address:SCM_PKG.git SCM_PKG' --scm-option spec='SCM_PKG.spec' --scm-option branch=1-2 --scm-option write_tar=True -v

scm-enable – turns on the use of scm
scm-option – set an option for the scm in use

The above worked for me, you will need to adjust it acordingly, i.e. if your spec file is not named identically to that of your git project: –scm-option spec=’specfile_name.spec’

This will tie me over untill I get chance to play with my monkey farmTweet

Gluster resolving a split brain in a replicated setup

Buzz — Tue, 20 Dec 2011 12:29:08 +0000

Initially this took about ~7hours to diagnose and fix, with what I have learned about the inner workings of gluster and the tools I am providing opensource this should cut resolution time down to ~5minutes.

Firs you must meet the following conditions:

You are running gluster >= 3.0 <= 3.2 (May also work on 2.x I have not tested, and will not work with future versions if gluster change their use of xattrs)
You are running a replicated volume (Again I have not tested distributed volumes, in theory remove, re-add and rebalance will fix these)
You have a “good” copy of you data (This is essential this assume you have at least 1 brick with a good copy of the file system

Restrain and restore the “bad” brick

Shutdown all services that are using the mounted filesystem (i.e. httpd / nginx / *ftpd)
Unmount all the file systems on the node (glusterfs / nfs / etc …)
Grab a copy of stripxattr.py make sure you READ the README for installation requirements and usage
Run stripxattr.py against the backing filesystem on the “bad” node ONLY NOT AGAINST A GLUSTER MOUNT
From the “good” node, not rsync the data: rsync -gioprtv –progress /path/to/filesystem root@:/path/to
From the “good” node, trigger an “auto heal” this will re-populate the xattr data (this must be done on a glusterfs mount not nfs/cifs/etc…)
Download listxattr.py once the self heal has completed see the README file for a “quick and dirty” consistency check
All being well you have now resolved a split-brain and can return your node to service

Current known gluster issues

NFS is much (48x in tests) faster for small files i.e. php webapps, but does not support distributed locking meaning: all nodes can write to the same file at the same time, this is what cause our original split brain

So what is the resolution int his case?

Selective use, use glusterfs for filesystems that you need distributed locking, often in large production deploys php files will not change often, in this case NFS is perfect.

If you are still writing php sessions to a file system then STOP IT and use a database! (Better yet use memcache).

An update. I know I haven’t been updating…

Buzz — Sun, 13 Nov 2011 13:53:37 +0000

I know I haven’t been updating a lot lately, esp on my poor blog (http://saiweb.co.uk), still I think I have things tied together enough to allow me to update once to everywhere (this post should appear on my blog, twitter, facebook, linkedin etc.

There’s been a lot developing over the last few months, Openstack being one of my main focuses along with overhauling and provision new internal systems for Openstack to run upon, I have a plan so to speak …

I have some Openstack posts coming I just need to ensure that all parties are happy with me posting the information “in the clear” so to speak.Tweet

Pivoting ssh reverse tunnel gateway

Buzz — Thu, 06 Oct 2011 14:43:02 +0000

They say necessity is the mother of invention, if this is true then surely the mother of all fuck ups is shoddy customer service, say an isp that will randomly shut down a port because it has high bandwidth usage without asking the customer about it first, and flat out refusing to do anything for 24hrs …

In one of the worst customer service experiences I’ve ever had the miss fortune to have been a part of all access was closed to our in office version control systems due to “high usage”, now this is a pretty essential service as you might imagine, how then to restore access, when the restrictions are beyond your control? (And I mean EVERY inbound port was dead …)

Fortunately it would seem outbound SSH was not affected, so after much vocal drawing and re-drawing many times over on the whiteboard I had a cunning plan …

Using 3 linux devices I would create the following.

1. A device through which using host entries / dns changes the version control would be available whilst not actually running on the box itself.

2. An in house device which would be the device on which the tunnels are created in the first place.

3. The device(s) on which the version control systems reside.

Gateway device

On the gateway device sshd_config needs to be updated with:

1	GatewayPorts yes

And sshd reloaded.

Also if you are using a local firewall (i.e. iptables) you will need to setup the appropriate rules as if the service were running natively on the device

Pivot Device

Generate rsa ssh keys and deploy your id_rsa.pub to the gateway device, (update sshd_config to enable RSA Auth if required)

The tunnel.

1	ssh -l root -g -N -R 0.0.0.0::10.0.0.1: -vvv

Now you only really need to use root if the port you need to gateway is a privileged port (<1024).

Here 10.0.0.1 is the internal address of the device the connection should "pivot" onto.

Once the tunnel was in place the services could be reached via the gateway device, this was essentially a "poor mans" NAT in a time of need, I really do not suggest this for long term use.

Linux collection of handy scripts and one liners – Volume 2 (Warning: contains shortcuts)

Buzz — Mon, 26 Sep 2011 15:43:34 +0000

See if hosts are up using ping in range 60 -> 200

1
2
3
4
5

for i in {60..200}; do ping -c 1 -W 1 192.168.1.$i > /dev/null; ([[ $? == 0 ]] && echo "$i UP" || echo "$i DOWN"); done
1 UP
2 DOWN
3 UP
...

Note: for OSX use “ping -c 1 -t 1″

Chaining “UP” hosts for a quick (syn) port scan

1
2
3

for i in {60..200}; do ping -c 1 -W 1 192.168.1.$i > /dev/null; ({{ $? == 0 ]] && nc -v -n -z -w1 192.168.1.$i 20-22); done
(UNKNOWN) [192.168.1.1] 22 (ssh) open
(UNKNOWN) [192.168.1.3] 22 (ssh) open

Recover from a bad mysql password set (Update mysql.users set password=’Iforgotawherestatemenlulz’)

Assumes for every user there is an @localhost host, grabs the in memory password hash and resets

mysql -Bse 'Select distinct(user) from mysql.user;' | while read uname; do mysql -Bse "show grants for '$uname'@'localhost';" 2>&1 | grep IDENTIFIED | grep -v 'root' | grep -v 'ERROR' | sed 's|GRANT USAGE ON *.* TO ||g' | sed "s|@'localhost' IDENTIFIED BY PASSWORD||g" | awk '{print "Update user set Password="$2" where User="$1";"}' | mysql mysql; done

If you’ve run FLUSH PRIVILEGES; however you == b0ned.

Quick substitute and run

Command1:

1	ping -c 1 -t 1 192.168.1.1

Opps that’s OSX synatx

Command2:

1	^-t 1^-W 1

et voila corrected syntax.

Shortcuts

!! – Execute last command
!ping – Execute last ping command, can be used to !any command just be careful.
ctrl+r – reverse search, just start typing the cmd for it to search your history, hit tab to complete
ctrl+a – jump to beginning of line
ctrl+e – jump to end of the line

cURL FU

curl -I -L blahblah.tld – Run a HEAD and follow redirects (very handy for quicklooking @ bit.ly short URLS before hitting them in a browser).

python FU

python -m SimpleHTTPServer – serves the current `pwd` as a browseable directory (Very cool but VERY insecure)
python -m cProfile script.py – generate trace stats for a script execution (Very handy for finding excessive loops)

DNS Fu

Wikipedia over DNS:

host -t txt fu.wp.dg.cx

fu.wp.dg.cx descriptive text “Fu may refer to: Fu (Technology, especially computer related) (used as a suffix) – relating to a person – Possessing superior skills in an art\; relating to an artifact – representing an expression of high art. code-fu, Perl-fu, C-fu, etc, Fu (literature),” ” a Chinese genre of rhymed prose, Fu (kana), a symbol in Japanese syllabaries, Fu County, in Shaanxi, China, Fu Foundation… http://a.vu/w:Fu”

Useful on _some_ public wifi connections if you just want to look something up quick (dns is not always re-written).

Get all MX servers for a domain:

dig google.co.uk MX

; <<>> DiG 9.6.0-APPLE-P2 <<>> google.co.uk MX
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 64165
;; flags: qr rd ra; QUERY: 1, ANSWER: 4, AUTHORITY: 4, ADDITIONAL: 4

;; QUESTION SECTION:
;google.co.uk. IN MX

;; ANSWER SECTION:
google.co.uk. 10800 IN MX 10 google.com.s9a1.psmtp.com.
google.co.uk. 10800 IN MX 10 google.com.s9a2.psmtp.com.
google.co.uk. 10800 IN MX 10 google.com.s9b1.psmtp.com.
google.co.uk. 10800 IN MX 10 google.com.s9b2.psmtp.com.

;; AUTHORITY SECTION:
google.co.uk. 59925 IN NS ns2.google.com.
google.co.uk. 59925 IN NS ns3.google.com.
google.co.uk. 59925 IN NS ns4.google.com.
google.co.uk. 59925 IN NS ns1.google.com.

;; ADDITIONAL SECTION:
ns1.google.com. 158334 IN A 216.239.32.10
ns2.google.com. 158334 IN A 216.239.34.10
ns3.google.com. 158741 IN A 216.239.36.10
ns4.google.com. 158334 IN A 216.239.38.10

;; Query time: 68 msec
;; SERVER:
;; WHEN: Mon Sep 26 16:41:26 2011
;; MSG SIZE rcvd: 310

mySQL FU

in one line, take a database, in stream replace content and stream into another db.

mysqldump original_db | sed ‘s/content_or_regex_to_replace/content_or_backref_replacement/g’ | mysql destination_db
Tweet

WiFi recon using OSX native tools

Buzz — Fri, 23 Sep 2011 10:13:12 +0000

So you wanted to get your aircrak suite on under OSX, getting airodump etc to work I can tell you will be a nightmare (infact just dont use a VM with a USB wifi for that, however there is an alternative …), after a lot of searching there is a native tool under OSX that will let you cap packets, list networks etc.

Credit goes to d3in0s for his awesome forum post.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120

/System/Library/PrivateFrameworks/Apple80211.framework/Versions/Current/Resources/airport
Usage: airport

If an interface is not specified, airport will use the first AirPort interface on the system.

prefs If specified with no key value pairs, displays a subset of AirPort preferences for
the specified interface.

Preferences may be configured using key=value syntax. Keys and possible values are specified below.
Boolean settings may be configured using 'YES' and 'NO'.

DisconnectOnLogout (Boolean)
JoinMode (String)
Automatic
Preferred
Ranked
Recent
Strongest
JoinModeFallback (String)
Prompt
JoinOpen
KeepLooking
DoNothing
RememberRecentNetworks (Boolean)
RequireAdmin (Boolean)
RequireAdminIBSS (Boolean)
RequireAdminNetworkChange (Boolean)
RequireAdminPowerToggle (Boolean)
WoWEnabled (Boolean)

logger Monitor the driver's logging facility.

sniff If a channel number is specified, airportd will attempt to configure the interface
to use that channel before it begins sniffing 802.11 frames. Captures files are saved to /tmp.
Requires super user privileges.

debug Enable debug logging. A debug log setting may be enabled by prefixing it with a '+', and disabled
by prefixing it with a '-'.

AirPort Userland Debug Flags
DriverDiscovery
DriverEvent
Info
SystemConfiguration
UserEvent
PreferredNetworks
AutoJoin
IPC
Scan
802.1x
Assoc
Keychain
RSNAuth
WoW
AllUserland - Enable/Disable all userland debug flags

AirPort Driver Common Flags
DriverInfo
DriverError
DriverWPA
DriverScan
AllDriver - Enable/Disable all driver debug flags

AirPort Driver Vendor Flags
VendorAssoc
VendorConnection
AllVendor - Enable/Disable all vendor debug flags

AirPort Global Flags
LogFile - Save all AirPort logs to /var/log/airport.log

is one of the following:
No options currently defined.

Examples:

Configuring preferences (requires admin privileges)
sudo airport en1 prefs JoinMode=Preferred RememberRecentNetworks=NO RequireAdmin=YES

Sniffing on channel 1:
airport en1 sniff 1

LEGACY COMMANDS:
Supported arguments:
-c[] --channel=[] Set arbitrary channel on the card
-z --disassociate Disassociate from any network
-I --getinfo Print current wireless status, e.g. signal info, BSSID, port type etc.
-s[] --scan=[] Perform a wireless broadcast scan.
Will perform a directed scan if the optional is provided
-x --xml Print info as XML
-P --psk Create PSK from specified pass phrase and SSID.
The following additional arguments must be specified with this command:
--password= Specify a WPA password
--ssid= Specify SSID when creating a PSK
-h --help Show this help
Credit goes to d3in0s post showing true forum awesomeness.



/System/Library/PrivateFrameworks/Apple80211.framework/Versions/Current/Resources/airport -I

     agrCtlRSSI: -40

     agrExtRSSI: 0

    agrCtlNoise: -92

    agrExtNoise: 0

          state: running

        op mode: station 

     lastTxRate: 54

        maxRate: 54

lastAssocStatus: 0

    802.11 auth: open

      link auth: wpa2-psk

          BSSID: 

           SSID: 

            MCS: -1

        channel: 6

/System/Library/PrivateFrameworks/Apple80211.framework/Versions/Current/Resources/airport -s

                            SSID BSSID             RSSI CHANNEL HT CC SECURITY (auth/unicast/group)

                            -41  6       N  -- WPA(PSK/AES,TKIP/TKIP) WPA2(PSK/AES,TKIP/TKIP)

Doing a frame cap.

1 2	/System/Library/PrivateFrameworks/Apple80211.framework/Versions/Current/Resources/airport en1 sniff 6 Capturing 802.11 frames on en1.

You will see your airport icon changes to now hit ctrl+c to stop the cap

1	^CSession saved to /tmp/airportSniff813ZrA.cap.

easy_install for python 3

Buzz — Mon, 19 Sep 2011 20:47:05 +0000

easy_install for python3 simple

1 2	curl -O http://python-distribute.org/distribute_setup.py python3 distribute_setup.py

Enjoy!Tweet

php mail() – Making it not suck using sendmail

Buzz — Sun, 18 Sep 2011 12:22:46 +0000

Ok ok … as some of the people work with are aware, I did this months ago fro one project, ment to blog and document it then in fact I have a draft post last modified 06/05/2011 covering full spam score reduction, and half finished instructions on setting up a mail relay … so in the interim of finishing that post I’m going to cover improving user experience through proper php configuration.

Out of the box, php will use sendmail, and it will do so as follows.

mail() forks sendmail process
sendmail attempts to send email to destination server
sendmail returns on send complete

Generally this isn’t a problem but what if at point 2. there is an issue with the destination MTA ? well in that case php will infact sit around waiting fot sendmail to complete, leaving your user with a hung screen / hung ajax call.

So what to do?

Simply put you want to offset the sending email process you do not want the end user sat around waiting for sendmail to finish sending the email, but you do want the email to send … decisions … decisions.

So edit yout php.ini .

1	sendmail_path = /usr/sbin/sendmail -t -i -O DeliveryMode=b

This sets the delivery mode to background, sendmail will return to php near instantly and send the email in the background by placing in into a queue.

TL;DR

Put the above in your php.ini to not hang around to sendmail, and hav it return instantly.
Tweet

PenTesting – mySQL password hash generation and lookup

Buzz — Wed, 17 Aug 2011 12:20:04 +0000

One of the worst things you can have in any secure system is a user with a simple password, no matter what steps you take to protect your data, if a privileged user is using a simple password, it’s akin to having a safe door that’s glass window.

First off we need to take a hash dump:

1	mysql -Bse 'select distinct(password),user from mysql.user;' > hashdump.txt

Now of course you can do the same using SQL Injection etc (WHY when you have SQLi already? duh privilege escalation!) , I’m going to cover this from the perspective that you are the administrator looking to strengthen your security …

Now you have your hashdump you need a hash table with the equivelent passwords within it, for this you will need 2 things

1. A dictionary file
2. https://github.com/Oneiroi/PenTesting/blob/master/crypto/generators/mysql/csv_gen.py

The python script above I wrote to use multiprocessing to map words onto the hash function, and I have had it grind through mySQL hashes at a rate of ~98k per second, there is no “lookup” script at this time though one is currently being written.

1	./csv_gen.py -f /path/to/wordlist.txt -o /output/path/to/output.csv -t [-l optional use legacy hash]

once this has ground through your wordlist you will have a CSV file, which will be in the format , the script defaults to the new PASSWORD() function, if you are using old_password=1 in your configuration then pass the -l flag to use legacy hashing instead.

ok let’s assume the following fictional scenario

1. old_passwords is in use, and we want chip’s password
2. 077b91e3491e2fdd chip
3.

1 2	grep 077b91e3491e2fdd output.txt 077b91e3491e2fdd,a

4. Chip has a password that is just he letter “a” which he will tell you is the best password ever …

And that’s about a simple as it gets you generate a set of hashes and you compare known hashes to your generate set to see if you can discern simple passwords, hopefully going on then to chastise the user and instructing them on proper password etiquette, there are more complicated methods of getting the password from the hash, in the case of old_passwords I believe it is possible to reverse the hash to get the original string for one (so don’t use old_passwords!)

If you go on to use my python scripts, please let me know how they perform, my test were carried out using an intel i5, I’d love to know how they perform on other CPUs.

Boxgrinder – setting up a simple CentOS LAMP stack, and deploying it to KVM

Buzz — Tue, 16 Aug 2011 11:35:54 +0000

If you haven’t tried boxgrinder then you are missing out, it makes it extremely easy to script the generation of a virtual machine for output to Rackspace (Well not yet), ec2, vmware, virtualbox, KVM etc.

In this post I will cover the basic generation of a LAMP (Linux Apache MySQL PHP) stack CentOS appliance, nothing to complicated I assure you, and no magic like auto deployment spin up etc … that’s for later … no skipping ahead!

First of all you’re going to need boxgrinder I recommend downloading the Meta appliance, as it has all the tools you need already.

Now I am covering the following.

basic use of boxgrinder-build on the meta appliance
creation of centos lampstack basic
deploying the image to KVM

I’m going to have to assume that you are capable of downloading and starting up the meta appliance yourself, and focus more on the stack setup.

Grinding your VM

Ok so you are going to need a YAML file defining the CentOS lamp stack, save this on your meta appliance as CentOS-lamp.yaml

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

name: CentOS-lamp
summary: Generic CentOS 5.6 LAMP stack, with some apache & php tuning
version: 1
release: 0
hardware:
cpus: 2
memory: 1024
partitions:
"/":
size: 5
"/var/www":
size: 15
os:
name: centos
version: 5
password: changeme

On your Meta appliance run.

1	boxgrinder-build -d CentOS-lamp.appl

This process will take a while, so go and get a coffee, this will produce ./build/appliances/x86_64/centos/5/CentOS-lamp/CentOS-lamp-sda.raw once complete, if you run into issues the -d flag is “debug” paste your log output int the comments and I will do my best to diagnose and fix your issue.

Deploying to KVM

boxgrinder has SFTP support for pushing to remote servers, you can use this if you like to automate the “push” of the image to your KVM server, at the moment automated deployment to KVM is not support but may be coming soon.

Assuming you have placed you image in /var/lib/libvirt/images/

1	virt-install -n "Saiweb - CentOS-lamp Demo" -r 1024 --arch=x86_64 --vcpus=1 --os-type=linux --os-variant=rhel5.4 --disk path=/var/lib/libvirt/images/CentOS-lamp.raw,size=20,cache=none,device=disk --accelerate --network=bridge:br0 --vnc --import

Post startup

this is a VERY basic setup I have not covered any of the post install options in this post (but I will in future posts), so.

1 2	chkconfig httpd on && service httpd start chkconfig mysqld on && service mysqld start

This will set your services to automatically start at startup, and start them.Tweet