Saiweb

PHP mail() not working?

getting “sh: -t: command not found” when testing using the cli?
what you have is a missing devel package!!!!

In my case sendmail-devel was missing, you’d think the configure script would alert on this but alas no, devel pack installed and one recompile later and the issue is solved.

Title for a cheesy sysadmin novel I know.

But over the last month or so I have been plagued by a mySQL server that was reporting a full root partition, when it wasn’t full …

Causing me some headaches

Allow me to explain:

Looks simple enough right? I just need to free up some space?

Afraid not.

Just for some clarification this small partition is in use for the operating system only, the mysql instance itself is infact mounted on a much larger partition using the same method as detailed in mysql moving /var/lib/mysql and error121

So here’s the problem, df and therfor all the monitoring systems are reporting the disk as full where as du clearly shows it is not …

Leaving me in the position of if I can not find where the disk usage is with du I have no way of freeing the disk space and bringing the service back online …

Or do I?

After talking with Matthew Ife of ukfast he suggests there must be a an unclosed file IO (aka a file descriptor) that is using up the diskspace, these descriptors do not show up using du

After some searching around I find the command lsof this command will list the open file descriptors for a process including their current size …

As you can see above the open file descriptor flagged as (deleted) was increasing in size until the diskspace ran out, for the time being I have since disabled mysql slow query logging whilst I sort out the log rolling as described in Mysql slow query log rotation