Saiweb

The mock tool can be a wonderful thing, allowing you to produce rpm packages for any rpm based system (assuming your have the written .cfg for it).

What I did find a little lacking on the documentation side was the SCM integration (read: Source Control Management), git/svn etc …

In short so long as your rpm spec file is in your SCM (and it should be), moc will build your rpm from your sources in scm, which can be used for.

1. bleeding edge builds for testing
2. builds from “stable tags”

Yes yes yes … obvious I know …

So with no futher ado here is the syntax:

1. scm-enable – turns on the use of scm
2. scm-option – set an option for the scm in use

The above worked for me, you will need to adjust it acordingly, i.e. if your spec file is not named identically to that of your git project: –scm-option spec=’specfile_name.spec’

This will tie me over untill I get chance to play with my monkey farmTweet

Initially this took about ~7hours to diagnose and fix, with what I have learned about the inner workings of gluster and the tools I am providing opensource this should cut resolution time down to ~5minutes.

Firs you must meet the following conditions:

1. You are running gluster >= 3.0 <= 3.2 (May also work on 2.x I have not tested, and will not work with future versions if gluster change their use of xattrs)
2. You are running a replicated volume (Again I have not tested distributed volumes, in theory remove, re-add and rebalance will fix these)
3. You have a “good” copy of you data (This is essential this assume you have at least 1 brick with a good copy of the file system

Restrain and restore the “bad” brick

1. Shutdown all services that are using the mounted filesystem (i.e. httpd / nginx / *ftpd)
2. Unmount all the file systems on the node (glusterfs / nfs / etc …)
3. Grab a copy of stripxattr.py make sure you READ the README for installation requirements and usage
4. Run stripxattr.py against the backing filesystem on the “bad” node ONLY NOT AGAINST A GLUSTER MOUNT
5. From the “good” node, not rsync the data: rsync -gioprtv –progress /path/to/filesystem root@:/path/to
6. From the “good” node, trigger an “auto heal” this will re-populate the xattr data (this must be done on a glusterfs mount not nfs/cifs/etc…)
7. Download listxattr.py once the self heal has completed see the README file for a “quick and dirty” consistency check
8. All being well you have now resolved a split-brain and can return your node to service

Current known gluster issues

1. NFS is much (48x in tests) faster for small files i.e. php webapps, but does not support distributed locking meaning: all nodes can write to the same file at the same time, this is what cause our original split brain

So what is the resolution int his case?

Selective use, use glusterfs for filesystems that you need distributed locking, often in large production deploys php files will not change often, in this case NFS is perfect.

If you are still writing php sessions to a file system then STOP IT and use a database! (Better yet use memcache).

Tweet

I know I haven’t been updating a lot lately, esp on my poor blog (http://saiweb.co.uk), still I think I have things tied together enough to allow me to update once to everywhere (this post should appear on my blog, twitter, facebook, linkedin etc.

There’s been a lot developing over the last few months, Openstack being one of my main focuses along with overhauling and provision new internal systems for Openstack to run upon, I have a plan so to speak …

I have some Openstack posts coming I just need to ensure that all parties are happy with me posting the information “in the clear” so to speak.Tweet

They say necessity is the mother of invention, if this is true then surely the mother of all fuck ups is shoddy customer service, say an isp that will randomly shut down a port because it has high bandwidth usage without asking the customer about it first, and flat out refusing to do anything for 24hrs …

In one of the worst customer service experiences I’ve ever had the miss fortune to have been a part of all access was closed to our in office version control systems due to “high usage”, now this is a pretty essential service as you might imagine, how then to restore access, when the restrictions are beyond your control? (And I mean EVERY inbound port was dead …)

Fortunately it would seem outbound SSH was not affected, so after much vocal drawing and re-drawing many times over on the whiteboard I had a cunning plan …

Using 3 linux devices I would create the following.

1. A device through which using host entries / dns changes the version control would be available whilst not actually running on the box itself.

2. An in house device which would be the device on which the tunnels are created in the first place.

3. The device(s) on which the version control systems reside.

Gateway device

On the gateway device sshd_config needs to be updated with:

And sshd reloaded.

Also if you are using a local firewall (i.e. iptables) you will need to setup the appropriate rules as if the service were running natively on the device

Pivot Device

Generate rsa ssh keys and deploy your id_rsa.pub to the gateway device, (update sshd_config to enable RSA Auth if required)

The tunnel.

Now you only really need to use root if the port you need to gateway is a privileged port (<1024).

Here 10.0.0.1 is the internal address of the device the connection should "pivot" onto.

Once the tunnel was in place the services could be reached via the gateway device, this was essentially a "poor mans" NAT in a time of need, I really do not suggest this for long term use.

Tweet

See if hosts are up using ping in range 60 -> 200

Note: for OSX use “ping -c 1 -t 1″

Chaining “UP” hosts for a quick (syn) port scan

Recover from a bad mysql password set (Update mysql.users set password=’Iforgotawherestatemenlulz’)

Assumes for every user there is an @localhost host, grabs the in memory password hash and resets

If you’ve run FLUSH PRIVILEGES; however you == b0ned.

Quick substitute and run

Command1:

Opps that’s OSX synatx

Command2:

et voila corrected syntax.

Shortcuts

!! – Execute last command
!ping – Execute last ping command, can be used to !any command just be careful.
ctrl+r – reverse search, just start typing the cmd for it to search your history, hit tab to complete
ctrl+a – jump to beginning of line
ctrl+e – jump to end of the line

cURL FU

curl -I -L blahblah.tld – Run a HEAD and follow redirects (very handy for quicklooking @ bit.ly short URLS before hitting them in a browser).

python FU

python -m SimpleHTTPServer – serves the current `pwd` as a browseable directory (Very cool but VERY insecure)
python -m cProfile script.py – generate trace stats for a script execution (Very handy for finding excessive loops)

DNS Fu

Wikipedia over DNS:

host -t txt fu.wp.dg.cx

fu.wp.dg.cx descriptive text “Fu may refer to: Fu (Technology, especially computer related) (used as a suffix) – relating to a person – Possessing superior skills in an art\; relating to an artifact – representing an expression of high art. code-fu, Perl-fu, C-fu, etc, Fu (literature),” ” a Chinese genre of rhymed prose, Fu (kana), a symbol in Japanese syllabaries, Fu County, in Shaanxi, China, Fu Foundation… http://a.vu/w:Fu”

Useful on _some_ public wifi connections if you just want to look something up quick (dns is not always re-written).

Get all MX servers for a domain:

dig google.co.uk MX

; <<>> DiG 9.6.0-APPLE-P2 <<>> google.co.uk MX
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 64165
;; flags: qr rd ra; QUERY: 1, ANSWER: 4, AUTHORITY: 4, ADDITIONAL: 4

;; QUESTION SECTION:
;google.co.uk. IN MX

;; ANSWER SECTION:
google.co.uk. 10800 IN MX 10 google.com.s9a1.psmtp.com.
google.co.uk. 10800 IN MX 10 google.com.s9a2.psmtp.com.
google.co.uk. 10800 IN MX 10 google.com.s9b1.psmtp.com.
google.co.uk. 10800 IN MX 10 google.com.s9b2.psmtp.com.

;; AUTHORITY SECTION:
google.co.uk. 59925 IN NS ns2.google.com.
google.co.uk. 59925 IN NS ns3.google.com.
google.co.uk. 59925 IN NS ns4.google.com.
google.co.uk. 59925 IN NS ns1.google.com.

;; ADDITIONAL SECTION:
ns1.google.com. 158334 IN A 216.239.32.10
ns2.google.com. 158334 IN A 216.239.34.10
ns3.google.com. 158741 IN A 216.239.36.10
ns4.google.com. 158334 IN A 216.239.38.10

;; Query time: 68 msec
;; SERVER:
;; WHEN: Mon Sep 26 16:41:26 2011
;; MSG SIZE rcvd: 310

mySQL FU

in one line, take a database, in stream replace content and stream into another db.

mysqldump original_db | sed ‘s/content_or_regex_to_replace/content_or_backref_replacement/g’ | mysql destination_db
Tweet

So you wanted to get your aircrak suite on under OSX, getting airodump etc to work I can tell you will be a nightmare (infact just dont use a VM with a USB wifi for that, however there is an alternative …), after a lot of searching there is a native tool under OSX that will let you cap packets, list networks etc.

Credit goes to d3in0s for his awesome forum post.

Doing a frame cap.

You will see your airport icon changes to now hit ctrl+c to stop the cap

Tweet

easy_install for python3 simple

Enjoy!Tweet

Ok ok … as some of the people work with are aware, I did this months ago fro one project, ment to blog and document it then in fact I have a draft post last modified 06/05/2011 covering full spam score reduction, and half finished instructions on setting up a mail relay … so in the interim of finishing that post I’m going to cover improving user experience through proper php configuration.

Out of the box, php will use sendmail, and it will do so as follows.

1. mail() forks sendmail process
2. sendmail attempts to send email to destination server
3. sendmail returns on send complete

This sets the delivery mode to background, sendmail will return to php near instantly and send the email in the background by placing in into a queue.

TL;DR

Put the above in your php.ini to not hang around to sendmail, and hav it return instantly.
Tweet

One of the worst things you can have in any secure system is a user with a simple password, no matter what steps you take to protect your data, if a privileged user is using a simple password, it’s akin to having a safe door that’s glass window.

First off we need to take a hash dump:

Now of course you can do the same using SQL Injection etc (WHY when you have SQLi already? duh privilege escalation!) , I’m going to cover this from the perspective that you are the administrator looking to strengthen your security …

Now you have your hashdump you need a hash table with the equivelent passwords within it, for this you will need 2 things

1. A dictionary file
2. https://github.com/Oneiroi/PenTesting/blob/master/crypto/generators/mysql/csv_gen.py

The python script above I wrote to use multiprocessing to map words onto the hash function, and I have had it grind through mySQL hashes at a rate of ~98k per second, there is no “lookup” script at this time though one is currently being written.

once this has ground through your wordlist you will have a CSV file, which will be in the format , the script defaults to the new PASSWORD() function, if you are using old_password=1 in your configuration then pass the -l flag to use legacy hashing instead.

ok let’s assume the following fictional scenario

1. old_passwords is in use, and we want chip’s password
2. 077b91e3491e2fdd chip
3.

4. Chip has a password that is just he letter “a” which he will tell you is the best password ever …

And that’s about a simple as it gets you generate a set of hashes and you compare known hashes to your generate set to see if you can discern simple passwords, hopefully going on then to chastise the user and instructing them on proper password etiquette, there are more complicated methods of getting the password from the hash, in the case of old_passwords I believe it is possible to reverse the hash to get the original string for one (so don’t use old_passwords!)

If you go on to use my python scripts, please let me know how they perform, my test were carried out using an intel i5, I’d love to know how they perform on other CPUs.

Tweet

If you haven’t tried boxgrinder then you are missing out, it makes it extremely easy to script the generation of a virtual machine for output to Rackspace (Well not yet), ec2, vmware, virtualbox, KVM etc.

In this post I will cover the basic generation of a LAMP (Linux Apache MySQL PHP) stack CentOS appliance, nothing to complicated I assure you, and no magic like auto deployment spin up etc … that’s for later … no skipping ahead!

First of all you’re going to need boxgrinder I recommend downloading the Meta appliance, as it has all the tools you need already.

Now I am covering the following.

1. basic use of boxgrinder-build on the meta appliance
2. creation of centos lampstack basic
3. deploying the image to KVM

I’m going to have to assume that you are capable of downloading and starting up the meta appliance yourself, and focus more on the stack setup.

Grinding your VM

Ok so you are going to need a YAML file defining the CentOS lamp stack, save this on your meta appliance as CentOS-lamp.yaml

On your Meta appliance run.

This process will take a while, so go and get a coffee, this will produce ./build/appliances/x86_64/centos/5/CentOS-lamp/CentOS-lamp-sda.raw once complete, if you run into issues the -d flag is “debug” paste your log output int the comments and I will do my best to diagnose and fix your issue.

Deploying to KVM

boxgrinder has SFTP support for pushing to remote servers, you can use this if you like to automate the “push” of the image to your KVM server, at the moment automated deployment to KVM is not support but may be coming soon.

Assuming you have placed you image in /var/lib/libvirt/images/

Post startup

this is a VERY basic setup I have not covered any of the post install options in this post (but I will in future posts), so.

This will set your services to automatically start at startup, and start them.Tweet