LinuxJedi's /dev/null

Previously I have shown my preferred methods of rotating log files here and here.  But Apache has a couple hidden gotchas and enhancements that can be made relating to logs.

Firstly the logs are not in true chronological order, the time stamps are at the start of the request, whilst the log data is actually recorded when the request finishes.  This means that long requests could be logged after short requests and the time stamps will be out of sequence.  This can mess with some log processors such as webalizer, which will drop out of sequence requests by default.  I have created a small utility called logsort which sorts out of order log files using minimal resources.

Another hiccup with Apache logging is that since 2.0 it logs how much the data it would send regardless of whether or not the transaction completes or not.  This can be problematic if you trying to figure out how much bandwidth a client is using based on the log files.  Luckily there is a work around in the name of mod_logio.  Simply enable mod_logio (in most Linux distros this means un-commenting it in httpd.conf) then look for your log line which will be something like:

LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined

and change it to:

LogFormat "%h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" combined

This will also include headers transferred.  You could also do this to record the amount of data recieved too (for example if there is a lot of binary post data):

LogFormat "%h %l %u %t \"%r\" %>s %I/%O \"%{Referer}i\" \"%{User-Agent}i\"" combined

This will produce:

[24/Feb/2008:14:19:07 +0000] "GET / HTTP/1.1" 200 104/7918 "" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.11) Gecko/20061201 Firefox/2.0.0.11 (Ubuntu-feisty)"

As a bonus you can also log the amount of time a request takes from start to completion (so processing time + transfer time) by doing:

LogFormat "%h %l %u %t \"%r\" %>s %I/%O \"%{Referer}i\" \"%{User-Agent}i\" %T/%D" combined

Which will give you:

[24/Feb/2008:14:19:07 +0000] "GET / HTTP/1.1" 200 104/7918 "" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.11) Gecko/20061201 Firefox/2.0.0.11 (Ubuntu-feisty)" 0/95491

Not content with just creating PHP APC packages for Ubuntu Hardy, I have also created updates for the current Memcached and PHP5 Memcache packages as they are a couple of versions old now.

Again these should re-compile on 32bit machines with ease and probably Gutsy too.

As some of you know I use Ubuntu Hardy (alpha) on my MacBook Pro.  Some think I am insane and twisted (and perhaps I am) but I find it much easier to do my day-to-day work on than Mac OSX.

Anyway, the one thing I found very strange about Ubuntu Hardy is that it favors XCache for an opcode cache without packages for eAccelerator or APC.  I personally use APC on all my servers for several reasons:

1. eAccelerator was segfaulting on our production servers after about a week of usage
2. At the time of evaluation XCache was still in its infancy (I'm sure its pretty good)
3. APC is a lot better than it used to be
4. APC is said to be included in PHP6

Anyway, if you want something done, do it yourself, and using that philosphy I have built a DEB for Ubuntu Hardy for APC which can be found in my downloads section.  It is currently for a 64bit install only but can be rebuilt for i386 and it should even rebuild on Gutsy.  I will try to keep the binary up to date as new versions of PHP are released on Hardy.

I just saw this and thought it was fantastic, nice one Sebastian!

See Sebastian's blog for the dialogue that go with it.

Sebastian Bergmann's Vader Testing picture

"Don't be too proud of this technological terror you've constructed. The ability to destroy bugs is insignificant next to the power of testing."

"Bugs do not concern me, Admiral. I want that code, not excuses."

etc...

Its too late, the damage is already done.  I say the word Linux and everybody (non-geek) knows I am talking about an operating system, I say GNU/Linux and they say "bless you" (not really, a poor attempt at a joke).

Its true what is said Brian's blog that it should be called GNU/Linux as just 'Linux' refers to the kernel.  But the public, my bosses, my bosses' bosses all know it as Linux.  This isn't down to me, this was planted in them long before I was on the scene.  Maybe I could reverse some of it, but other companies that they communicate to won't.

So whilst I agree it should be called GNU/Linux, I think the damage is already done and a GNU/Linux distribution will be called 'Linux' for some time to come.

Maybe I should be the GNU/LinuxJedi