In Part 1

I covered the use of pmap to provide an insight into the memory footprint of the apache service.

In Part 2

I covered a list modules that are typically enabled in a standard httpd/apache setup, and listed which modules I was disabling to reduce the memory foot print.

After improving the appmem script in Part 4 of the linux sysadmin script I updated part 3 of this series with the new memory foot print figures.

Summary aside, now we have reduced the memory footprint of your apache installation lets go onto actually making the most of your new found memory excess ..’potential’ max clients.

As stated in part 3 of this series, I am assuming the server you are making these configuration changes is a dedicated apache server.

However you also need to be aware of your CGI programs such as PHP, if you are to configure PHP with a maximum memory of 32MB, you must be aware that there is the potential for PHP to try to use this much memory on each allocation, this is where “optimizing PHP gor high load sites” comes into play, a furture series I plan to write, for the time being however I am only covering the configuration of apache.

Your httpd.conf should have the following entries (or similar).

As can be seen above in my installation the part of the config I am interested in is prefork.c.

Now with my theorehtical max clients in part 3 hitting 800.5, lets first go for a safer figure, say 60%

From here use the script in part 1 of the Linux sysadmin script, to monitor your active http connections, and ‘tweak’ your configuration accordingly.

Remember you must only set the MaxClients and ServerLimit to a value you can safely contain in ram, swapping out to disk will cause a major slow down.Tweet

• LoadModule auth_basic_module modules/mod_auth_basic.so

ENABLED: User authentication for access control using HTTP basic authentication (.htaccess and .htpasswd)

• LoadModule auth_digest_module modules/mod_auth_digest.so

ENABLED: Similar to auth_basic_module but instead of using a plain text authentication scheme, it uses a cryptographic one. (MD5 etc …)

• LoadModule authn_file_module modules/mod_authn_file.so

ENABLED: Allows authentication front-ends such as auth_digest_module and auth_basic_module to authenticate users by looking up users in plain text password files. This function was previously part of auth_module and auth_digest_module.

• LoadModule authn_alias_module modules/mod_authn_alias.so

DISABLED: Allows extended authentication providers to be created within the configuration file and assigned an alias name.

• LoadModule authn_anon_module modules/mod_authn_anon.so

DISABLED: Allows anonymous user access and logs the password given. Previously known asauth_anon_module.

• LoadModule authn_dbm_module modules/mod_authn_dbm.so

DISABLED: Allows authentication front-ends such as auth_digest_module and auth_basic_module to authenticate users by looking up users in dbm password files. Previously known asauth_dbm_module.

• LoadModule authn_default_module modules/mod_authn_default.so

DISABLED: Fallback authentication module – it simply rejects any credentials supplied by the user

• LoadModule authz_host_module modules/mod_authz_host.so

ENABLED: Access control by browser hostname. Previously known asaccess_module.

• LoadModule authz_user_module modules/mod_authz_user.so

DISABLED: Provides authorization capabilities so that authenticated users can be allowed or denied access to portions of the web site. This function was previously part of auth_module.

• LoadModule authz_owner_module modules/mod_authz_owner.so

DISABLED: Authorization based on file ownership.

• LoadModule authz_groupfile_module modules/mod_authz_groupfile.so

DISABLED: Group authorization using plaintext files. This function was previously part of auth_module.

• LoadModule authz_dbm_module modules/mod_authz_dbm.so

DISABLED: Group authorization using DBM files. This function was previously part of auth_dbm_module.

• LoadModule authz_default_module modules/mod_authz_default.so

DISABLED: Fallback authorisation module – it simply rejects any authorization request.

• LoadModule ldap_module modules/mod_ldap.so

DISABLED: LDAP connection pooling and result caching services for use by other LDAP modules.

(I don’t use LDAP, if you use apache interaction with ldap keep this module, if you use LDAP within PHP you can turn this mod off!)

• LoadModule authnz_ldap_module modules/mod_authnz_ldap.so

DISABLED: Allows authentication front-ends such as auth_basic_module to authenticate users through an ldap directory. Previously known asauth_ldap_module. (same as above)

• LoadModule include_module modules/mod_include.so

DISABLED: Server-side includes.

• LoadModule log_config_module modules/mod_log_config.so

ENABLED: Configurable logging of requests and reponses.

• LoadModule logio_module modules/mod_logio.so

ENABLED: Logging of input and output bytes per request.

• LoadModule env_module modules/mod_env.so

DISABLED: Changes the environment that CGI program are run in.

• LoadModule ext_filter_module modules/mod_ext_filter.so

ENABLED: Pass the response body through an external program before delivery to the client (i’m using mods like gzip compression, disalbe this if you don’t need it).

• LoadModule mime_magic_module modules/mod_mime_magic.so

ENABLED: Determines MIME types based on file contents. (Great for directly serving download easily).

• LoadModule expires_module modules/mod_expires.so

ENABLED: Autogenerates the Expires: header according to user rules.

• LoadModule deflate_module modules/mod_deflate.so

ENABLED: Compress content prior to serving it. (If you don’t have compression configured for your websites disable this mod).

• LoadModule headers_module modules/mod_headers.so

ENABLED: More general control of HTTP headers. (You can disable this if you do not want to handle bespoke HTTP headers)

• LoadModule usertrack_module modules/mod_usertrack.so

DISABLED: Provision of cookies. (I am using PHP to handle cookies).

• LoadModule setenvif_module modules/mod_setenvif.so

ENABLED: Sets the environment for CGI programs based on properties of the request.

• LoadModule mime_module modules/mod_mime.so

ENABLED: Determines MIME types based on file names.

• LoadModule dav_module modules/mod_dav.so

DISABLED: Distributed Authoring and Versioning functionality. (This is a webserver not serving a versioning system).

• LoadModule status_module modules/mod_status.so

ENABLED: Provides information about the server’s current status via a web request. (If you don’t have this set in your httpd.conf then disable the mod, I use this for apache monitoring).

• LoadModule autoindex_module modules/mod_autoindex.so

ENADLED: Automatically picks the index file when browsing to a folder, (index.html, index.php etc ..)

• LoadModule info_module modules/mod_info.so

DISABLED: Similar to a php info call, provides running configuration information about the apache server.

• LoadModule dav_fs_module modules/mod_dav_fs.so

DISABLED: More no required D.A.V

• LoadModule vhost_alias_module modules/mod_vhost_alias.so

ENABLED: Allows for handling enormous numbers of virtual hosts without having to change the configuration each time.

• LoadModule negotiation_module modules/mod_negotiation.so

ENABLED: Provides for content negotiation between server and client. (Accept encoding etc…)

• LoadModule dir_module modules/mod_dir.so

ENABLED: Supports the use of index.html files for directory lookups.

• LoadModule actions_module modules/mod_actions.so

ENABLED: Run specific CGI programs according to the MIME content type of the object served.

• LoadModule speling_module modules/mod_speling.so

ENABLED: Attempts to correct misspelled URLs. (Ignores case etc..)

• LoadModule userdir_module modules/mod_userdir.so

ENABLED: Allow public_html userdir and use of http://servername/~username, I have disabled this as can be used to verify the presence of a valid username to be used in a brute force attack

• LoadModule alias_module modules/mod_alias.so

ENABLED: Override the DocumentRoot directive for specific URLs.

• LoadModule rewrite_module modules/mod_rewrite.so

ENABLED: Allows the use of the Rewriteengine to provide SEO friends URL’s amoungst other things.

• LoadModule proxy_module modules/mod_proxy.so

ENABLED: Lets your web server be a proxy. Typically it needs additional modules for specific protocols.

• LoadModule proxy_balancer_module modules/mod_proxy_balancer.so
• LoadModule proxy_ftp_module modules/mod_proxy_ftp.so
• LoadModule proxy_http_module modules/mod_proxy_http.so
• LoadModule proxy_connect_module modules/mod_proxy_connect.so

ENABLED: As above, I have need to have apache act as a proxy, in the future

• LoadModule cache_module modules/mod_cache.so

ENABLED: Implements an RFC 2616 compliant HTTP content cache that can be used to cache either local or proxied content.

• LoadModule suexec_module modules/mod_suexec.so

ENABLED: Allows CGI scripts to run as a user other than wwwrun.

• LoadModule disk_cache_module modules/mod_disk_cache.so
• LoadModule file_cache_module modules/mod_file_cache.so
• LoadModule mem_cache_module modules/mod_mem_cache.so

ENABLED: More cache-ing functions (reduces need for disk read time).

• LoadModule cgi_module modules/mod_cgi.so

ENABLED: Run CGI programs

• LoadModule version_module modules/mod_version.so

ENABLED: Version dependent configuration (Can be handy when writing configs to have this enabled).

Source’s used:

Module information can be found HERE: http://www-uxsup.csx.cam.ac.uk/~jw35/courses/apache/html/a2617.htm

So 143mb down from 189mb, saving 46mb/thread.

Not much of a saving right? … well actually wrong.

Take for example a webserver with 2048MB Avail system ram, lets allocate 15% of that for the OS use.

So we have 1741MB to play with, with the original setup of 189mb 9.21 concurrent connections could be active an any one given time.

now however 12.71 concurrent connections can be in memory at any given time (28% increase).

Still a long way to go to bring the memory foot print down to a minimal level, but as you can see spending 5 minutes disabling mod’s in the config yields a quick win 28% increase in capacity.

UPDATE

Following the improvements to the appmem script in part 4 here are the new figures.

With all mods enabled:

Results attained running “ab -n 1000 -c 100 http://xxx.xxx.xxx.xxx/”.

Results with disabled mods (listed above):

Shared mem down 36MB (20.45% improvement), RSS/PID down 1mb (33% improvement)

Allowing for 1741MB availble ram, means 1601mb (1741MB -140MB shared) is available for apache threads, allowing for 800.5 concurrent connections with the improved config and by comparison

1565 MB Avail for threads (1741MB – 176MB), allowing 521.66 concurrent connections.

So with the improvements outlined above in theory the Apache server can now handle ~280 more concurrent connections.

Tweet

So why would you want to reduce the memory footprint?

Simple really so you can have more concurrent connections to the apache server without having to constantly keep adding more memory to your web server.

This ‘tutorial’ does assume the following:

• You are running apache on a server dedicated to doing so.
• You have a basic understanding of how to install and configure apache.
• You have a basic understanding of bash scripting.
• You have the pmap package installed on your server.

As was introduced in part-3 the function appmem will report the memory usage of all current running threads of an application name, report the average memory usage, and an example pmap command using the last PID in the list.

This is where this entry will pick up, once pmap is run you will see the following example output:

As can be seen above via the script and pmap output, the majority of apache’s memory footprint is taken up by dynamicaly loaded libraries or ‘mods’.

Take for example mod_authz_ldap, this module itself on each thread spawn (new connection) is taking up ~2mb of physical memory space, now this may not seem like much until you are serving 100′s of concurrent connections, at which point this becomes a very real problem.

mod_authz_ldap I have used as an example because it is an authentication module for use with the Lightweight Directory Access Protocol, something not commonly used in a generic LAMP setup.

By going through your httpd.conf and disabling modules you know you do not require (such as the example above), you can substantially reduce the memory footprint of apache, and therefor increase the number of concurrent connections apache can serve with the available memory.

In part 2 I will provide a new ouput of the script and pmap post optimization … stay tuned Tweet

In part 3, I am going to cover a bash function that will allow you to profile the memory usage of any application by name.

By adding the function below into your script you can execute a command such as: sysadmin appmem apache

Sample output:

You can of course replace ‘apache’ with the application or daemon name you want to profile the memory usage of.

This script does require that pmap is installed, if the script can not find it, it will abort.

As always any problems, post a comment.

UPDATE: Apparently I need to point out that if you haven’t read PART 2! then the colored output will not work … That’s why this entry is titled part 3, it does assume a degree of competence on your part in realizing part’s 1 and 2 may just be required reading …

NOTE: The above provides a complete memory footprint of the indvidual PID, the same as VIRT in top.

VIRT — Virtual Image (kb)
* The total amount of virtual memory used by the task. It includes all code, data and shared libraries plus pages that have been swapped out.
* VIRT = SWAP + RES Tweet

In this part I will be covering the concepts of secured shared hosting, and why you as a shared hosting provider should be taking steps to ensure this is how you deploy your hosting environments.

Let’s first take a typical L.A.M.P setup:

PHP Compiled from source as apache module.
mySQL installed from RPM or update package (yum / up2date).
HTTPD installed as RPM or update package (yum / up2date).

Please note at the time of writing if you yum / apt-get / up2date install your PHP package you will have varying results when attempting to compile and install suPHP, as such grab the source code from php.net, and follow this series.

As a shared hosting provider lets say you have 5 clients all hosted from the one server, each client using vsftpd is chrooted() into their home directory, and their ssh access disabled, supposedly secure enough.

Unfortunatly not so, due to the L.A.M.P configuration the ‘apache’ user needs a minimum of read and execute permissions over all the PHP files on the system, why is this a problem?

This is a problem largely due to human nature of the client, your ‘joe bloggs’ client doesn’t care about the technical aspects of web hosting or websites, they just want an easy pretty interface to get their corner of the internet online, downloading something like drupal or joomla.

Now this isn’t a dig at open source CMS, this is an insight into human nature, look at the changelog for any open CMS and you will see ‘security fixes’, unfortunatly all ‘joe bloggs’ cares about is that their website is working, and this is wher things take a turn for the worse.

Joe Bloggs never updates his open CMS platform, meaning any vulnerabilities patched in subsequent releases are still exploitable on his website, worst case scenario that this is an XSSI (Cross Server Script Includes) vulnerbility.

An attacker finds this website and idetifies the security hole, using XSSI to install a PHP interactive shell, giving the attacker SSH like access to the hosting environment, most people at this point think so the attacker has compromise one site … so what we can restore that site from backups and it’s only one site that’s affected, the other 4 users either do not use open CMS or are up to date with all the security patches.

Well that’s where you would be wrong, with the hosting setup outlined above the SSH like PHP shell is now running as the apache user, meaning the attacker can go anywhere and read anything apache can, and with the hosting setup oulined above that mean reading things like datbase connection files, suddenly all the clients on the hosting environment have their websites compromised as the attacker gains mySQL access and starts changing content on thewebsites, despite the fact that the other 4 sites themselves were never exploited.

One clients error just became a cascading exploit on your hosting platform, now make that a more realistic platform say 30 clients on the box, some are online shops, the issue just became a whole lot bigger there is lost revenue due to downtime of the shop sites, and worse still the attacker now has access to any customer details those shops were storing! but it’s not Joe Bloggs that’s accountable it’s YOU as the hosting provider, you can take steps to prevent one exploited site becoming 30, and this web series will tell you host to do it.

coming in part 2:

an introduction to suPHP
compiling php as a cgi binary, and why you need to do so

Tweet

Sould I participate with http://no-www.org/ and remove support for www. on my sites?

The answer is no you shouldn’t whilst I appreciate and do belive www. has become deprecated, a LOT of end users still use www. So by removing support for this you are excludinga potentialy large user base from reaching your website, can you as a business afford to do that … of course not!

So what _should_ you do?

In my opinion, add support for both.

You will see that http://saiweb.co.uk and http://www.saiweb.co.uk BOTH work, this allows both “classes” of end users to reach my website, and I have applied this principle to all sites I have recently worked on.

Example httpd.conf entry

ServerName saiweb.co.uk
ServerAlias www.saiweb.co.uk

DNS for saiweb.co.uk is an A record to the IP of my webhost and www.saiweb.co.uk is a CNAME of saiweb.co.uk

This method however does not http://no-www.org/ validate as www. subdomain still exists, so don’t expect a “no-www” validated image to appear on my blog anytime soon.

www. may be depreciated, but businesses can not afford to exclude any potential customers, so don’t expect this no-www to become standard anytime either.

“July 2, 2008

Today we passed 38,000 domains validated through no-www. This is a nice round number so it seems worthy of tooting our horn over. ”

As I write this there are 77,343,623 active domains (source: http://www.domaintools.com/internet-statistics/) making the no-www support less than 0.05% of active domains.

I would be intrested to know how many of this 38k domains are actual businesses, and how support no-www has affected thier online sales.

I can tell you however of the 300 or so domains I currently controll, non would no-www validate as they are all configured using the method above.

Thoughts?Tweet

 In favour of an all singing all dancing NEW ONE! nativespace thus-far I have had excellent ticket turn around (all in 30 mins or less), and my initial sales enquiry (consisting of a lot of lengthy questions)  responded to in …. 6 minutes!

So thus far definitely on my recommended listTweet

 ”Hey I realy could do with this re-write rule on that site, but I don’t want it applying to all sites running on the same framework”

Well fear not, after much head scratching, AccessFileName directive to the rescue! i.e.

 Using the above method you can specify bespoke htaccess files on a per VirtualHost basis.

 Enjoy!Tweet