1. Reading log files with Tail and Awk

    Reading the logfiles of your box can be handy for different purposes. I use it to check if an application is able to send email, to check failed SSH login attempts, or e.g. to see how people react on a new registration form on one of my sites.

    Ok, so let’s check our mail logfile with the tail utility.
    tail -f /var/log/mail.log -n 100
    The -f flag stands for follow, and outputs appended data as the file grows. So while your mail server is sending emails out, you will see them appear on the screen. Next is the path to your log file, should be “/var/log/mail.log”. And the last part of the command “-n 100” outputs the last 100 lines, instead of the last 10 that are standard in the tail program.
    Now let’s check our Apache log files combining the Tail command from the previous example with Awk a pattern scanning and text processing language.
    tail -f /var/log/apache2/yoursitename.log -n 1000 | awk -F " " '{print $1 " " $11}'
    So, the first part of the command is the same as in the previous example, except for the fact that we are outputting the last 1000 lines of the log file (so we can track the visitors better). Then there is the vertical bar (pipeline), that tells the shell that the output of the tail command should be the input of the AWK program, that way we are manipulating the log file with two different programs, tail and AWK.
    AWK is line oriented, let’s see how a line in the Apache log file looks like: - - [01/Apr/2012:08:14:16 +0200] "GET /images/logo.jpg HTTP/1.1" 200 2333 "http://www.yoursite.com/aboutus.html" "Mozilla/5.0
    First we call the AWK program with the -F flag, that sets the field separator to be a whitespace (as you can see the fields or columns in the Apache log file are separated by a whitespace). So for each line in the whole log file the AWK program checks to see if there is a whitespace. If it finds one it creates a new field (column) which we can output to the terminal. This is what is happening in the last part of the command. As you can see i’m only interested in column one and eleven ( ‘{print $1 ” ” $11}’ ) of each line, the ip-address respectively the URL that people are on.

    Checking running queries on MySql
    Edit your MySQL config: /etc/mysql/my.cnf - look for, or add, a line like this:
    log = /var/log/mysql/mysql.log

    Now restart MySql:
    /etc/init.d/mysql restart

    Now use Tail again to see the queries:
    tail -f /var/log/mysql/mysql.log
    That’s it, hope you liked it,