Tuesday, January 31, 2012

20 Linux Log Files that are Located under /var/log Directory

If you spend lot of time in Linux environment, it is essential that you know where the log files are located, and what is contained in each and every log file.
When your systems are running smoothly, take some time to learn and understand the content of various log files, which will help you when there is a crisis and you have to look though the log files to identify the issue.

/etc/rsyslog.conf controls what goes inside some of the log files. For example, following is the entry in rsyslog.conf for /var/log/messages.
$ grep "/var/log/messages" /etc/rsyslog.conf
*.info;mail.none;authpriv.none;cron.none                /var/log/messages
In the above output,
  • *.info indicates that all logs with type INFO will be logged.
  • mail.none,authpriv.none,cron.none indicates that those error messages should not be logged into the /var/log/messages file.
  • You can also specify *.none, which indicates that none of the log messages will be logged.
The following are the 20 different log files that are located under /var/log/ directory. Some of these log files are distribution specific. For example, you’ll see dpkg.log on Debian based systems (for example, on Ubuntu).
  1. /var/log/messages – Contains global system messages, including the messages that are logged during system startup. There are several things that are logged in /var/log/messages including mail, cron, daemon, kern, auth, etc.
  2. /var/log/dmesg – Contains kernel ring buffer information. When the system boots up, it prints number of messages on the screen that displays information about the hardware devices that the kernel detects during boot process. These messages are available in kernel ring buffer and whenever the new message comes the old message gets overwritten. You can also view the content of this file using the dmesg command.
  3. /var/log/auth.log – Contains system authorization information, including user logins and authentication machinsm that were used.
  4. /var/log/boot.log – Contains information that are logged when the system boots
  5. /var/log/daemon.log – Contains information logged by the various background daemons that runs on the system
  6. /var/log/dpkg.log – Contains information that are logged when a package is installed or removed using dpkg command
  7. /var/log/kern.log – Contains information logged by the kernel. Helpful for you to troubleshoot a custom-built kernel.
  8. /var/log/lastlog – Displays the recent login information for all the users. This is not an ascii file. You should use lastlog command to view the content of this file.
  9. /var/log/maillog /var/log/mail.log – Contains the log information from the mail server that is running on the system. For example, sendmail logs information about all the sent items to this file
  10. /var/log/user.log – Contains information about all user level logs
  11. /var/log/Xorg.x.log – Log messages from the X
  12. /var/log/alternatives.log – Information by the update-alternatives are logged into this log file. On Ubuntu, update-alternatives maintains symbolic links determining default commands.
  13. /var/log/btmp – This file contains information about failed login attemps. Use the last command to view the btmp file. For example, “last -f /var/log/btmp | more”
  14. /var/log/cups – All printer and printing related log messages
  15. /var/log/anaconda.log – When you install Linux, all installation related messages are stored in this log file
  16. /var/log/yum.log – Contains information that are logged when a package is installed using yum
  17. /var/log/cron – Whenever cron daemon (or anacron) starts a cron job, it logs the information about the cron job in this file
  18. /var/log/secure – Contains information related to authentication and authorization privileges. For example, sshd logs all the messages here, including unsuccessful login.
  19. /var/log/wtmp or /var/log/utmp – Contains login records. Using wtmp you can find out who is logged into the system. who command uses this file to display the information.
  20. /var/log/faillog – Contains user failed login attemps. Use faillog command to display the content of this file.
Apart from the above log files, /var/log directory may also contain the following sub-directories depending on the application that is running on your system.
  • /var/log/httpd/ (or) /var/log/apache2 – Contains the apache web server access_log and error_log
  • /var/log/lighttpd/ – Contains light HTTPD access_log and error_log
  • /var/log/conman/ – Log files for ConMan client. conman connects remote consoles that are managed by conmand daemon.
  • /var/log/mail/ – This subdirectory contains additional logs from your mail server. For example, sendmail stores the collected mail statistics in /var/log/mail/statistics file
  • /var/log/prelink/ – prelink program modifies shared libraries and linked binaries to speed up the startup process. /var/log/prelink/prelink.log contains the information about the .so file that was modified by the prelink.
  • /var/log/audit/ – Contains logs information stored by the Linux audit daemon (auditd).
  • /var/log/setroubleshoot/ – SELinux uses setroubleshootd (SE Trouble Shoot Daemon) to notify about issues in the security context of files, and logs those information in this log file.
  • /var/log/samba/ – Contains log information stored by samba, which is used to connect Windows to Linux.
  • /var/log/sa/ – Contains the daily sar files that are collected by the sysstat package.
  • /var/log/sssd/ – Use by system security services daemon that manage access to remote directories and authentication mechanisms.
Instead of manually trying to archive the log files, by cleaning it up after x number of days, or by deleting the logs after it reaches certain size, you can do this automatically using logrotateas we discussed earlier.
To view the log files use any one of the following methods. But, please don’t do ‘cat | more’.
  • vi – If you are comfortable with the vi commands, use vi editor for quick log file browsing.
  • tail – If you want to view the content of the log files real time, as the application is writting to it, use “tail -f”. You can also view multiple log files at the same time (using “tail -f”).
  • grep – If you know exactly what you are looking for in a log file, you can quickly use grep command to grep a pattern. The 15 practical grep examples will take out all your excuses of not using grep.
  • less – Less command is very powerful to browse log files. Use these 10 less command tipsto master the less command.

How to Run Cron Every 5 Minutes, Seconds, Hours, Days, Months

Crontab can be used to schedule a job that runs on certain internal. The example here show how to execute a backup.sh shell script using different intervals.

Also, don’t forget to read our previous crontab article that contains 15 practical examples, and also explains about @monthly, @daily, .. tags that you can use in your crontab.

1. Execute a cron job every 5 Minutes

The first field is for Minutes. If you specify * in this field, it runs every minutes. If you specify */5 in the 1st field, it runs every 5 minutes as shown below.
*/5 * * * * /home/suresh/backup.sh
Note: In the same way, use */10 for every 10 minutes, */15 for every 15 minutes, */30 for every 30 minutes, etc.

2. Execute a cron job every 5 Hours

The second field is for hours. If you specify * in this field, it runs every hour. If you specify */5 in the 2nd field, it runs every 5 hours as shown below.
0 */5 * * * /home/suresh/backup.sh
Note: In the same way, use */2 for every 2 hours, */3 for every 3 hours, */4 for every 4 hours, etc.

3. Execute a job every 5 Seconds

Cron job cannot be used to schedule a job in seconds interval. i.e You cannot schedule a cron job to run every 5 seconds. The alternative is to write a shell script that uses ‘sleep 5′ command in it.
Create a shell script every-5-seconds.sh using bash while loop as shown below.
$ cat every-5-seconds.sh
while true
 sleep 5
Now, execute this shell script in the background using nohup as shown below. This will keep executing the script even after you logout from your session. This will execute your backup.sh shell script every 5 seconds.
$ nohup ./every-5-seconds.sh &

4. Execute a job every 5th weekday

This example is not about scheduling “every 5 days”. But this is for scheduling “every 5th weekday”.
The 5th field is DOW (day of the week). If you specify * in this field, it runs every day. To run every Friday, specify either 5 of Fri in this field.
The following example runs the backup.sh every Friday at midnight.
0 0 * * 5 /home/suresh/backup.sh
0 0 * * Fri /home/suresh/backup.sh
You can either user number or the corresponding three letter acronym for the weekday as shown below.
  • 0=Sun
  • 1=Mon
  • 2=Tue
  • 3=Wed
  • 4=Thu
  • 5=Fri
  • 6=Sat
Note: Get into the habit of using Fri instead of 5. Please note that the number starts with 0 (not with 1), and 0 is for Sun (not Mon).

5. Execute a job every 5 months

There is no direct way of saying ‘every 5 months’, instead you have to specify what specific months you want to run the job. Probably you may want to run the job on 5th month (May), and 10th month (Oct).
The fourth field is for Months. If you specify * in this field, it runs every month. To run for the specific month, you have to specify the number that corresponds to the month. For example, to run the job on May and Oct, you should specify 5,10 (or) you can simply use the 3 letter acronym of the month and specify May,Oct.
The third field is for DOM (Day of the Month). If you specify * in this field, it runs every day of the month. If you specify 1 in this month, it runs 1st of the month.
The following example runs the backup.sh twice a year. i.e 1st May at midnight, and 1st Oct at midnight.
0 0 1 5,10 * /home/suresh/backup.sh
0 0 1 May,Oct * /home/suresh/backup.sh
Note: Don’t make the mistake of specifying 5-10 in the 4th field, which means from 5th month until 10th month. If you want only 5th and 10th month, you should use comma.