1.5.1 Understanding the Audit Logs

The following examples highlight two typical events that are logged by audit and how their trails in the audit log are read. The audit log or logs (if log rotation is enabled) are stored in the /var/log/audit directory. The first example is a simple less command. The second example covers a great deal of PAM activity in the logs when a user tries to remotely log in to a machine running audit.

type=SYSCALL msg=audit(1175176190.105:157): arch=40000003 syscall=5 success=yes
exit=4 a0=bfba161c a1=8000 a2=0 a3=8000 items=1 ppid=4457 pid=4462 auid=0 
uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=pts0 comm="less" exe="/usr/bin/less" subj=unconstrained  key="LOG_audit_log"
type=CWD msg=audit(1175176190.105:157):  cwd="/tmp"
type=PATH msg=audit(1175176190.105:157): item=0 name="../var/log/audit/audit.log" inode=458325 dev=03:01 mode=0100640 ouid=0 ogid=0 rdev=00:00

The above event, a simple less /var/log/audit/audit.log, wrote three messages to the log. All of them are closely linked together and you would not be able to make sense of one of them without the others. The first message reveals the following information:

type

The type of event recorded. In this case, it assigns the SYSCALL type to an event triggered by a system call (less or rather the underlying open). The CWD event was recorded to record the current working directory at the time of the syscall. A PATH event is generated for each path passed to the system call. The open system call takes only one path argument, so only generates one PATH event. It is important to understand that the PATH event reports the pathname string argument without any further interpretation, so a relative path requires manual combination with the path reported by the CWD event to determine the object accessed.

msg

A message ID enclosed in brackets. The ID splits into two parts. All characters before the : represent a UNIX epoch time stamp. The number after the colon represents the actual event ID. All events that are logged from one application's system call have the same event ID. If the application makes a second system call, it gets another event ID.

arch

References the CPU architecture of the system call. Decode this information using the -i option on any of your ausearch commands when searching the logs.

syscall

The type of system call as it would have been printed by an strace on this particular system call. This data is taken from the list of system calls under /usr/include/asm/unistd.h and may vary depending on the architecture. In this case, syscall=5 refers to the open system call (see man open(2)) invoked by the less application.

success

Whether the system call succeeded or failed.

exit

The exit value returned by the system call. For the open system call used in this example, this is the file descriptor number. This varies by system call.

a0 to a3

The first four arguments to the system call in numeric form. The values of these are totally system call dependent. In this example (an open system call), the following are used:

a0=bfba161c a1=8000 a2=0 a3=8000

a0 is the start address of the passed pathname. a1 is the flags. 8000 in hex notation translates to 100000 in octal notation, which in turn translates to O_LARGEFILE. a2 is the mode, which, because O_CREAT was not specified, is unused. a3 is not passed by the open system call. Check the manual page of the respective system call to find out which arguments are used with it.

items

The number of strings passed to the application.

ppid

The process ID of the parent of the process analyzed.

pid

The process ID of the process analyzed.

auid

The audit ID. A process is given an audit ID on user login. This ID is then handed down to any child process started by the initial process of the user. Even if the user changes his identity (for example, becomes root), the audit ID stays the same. Thus you can always trace actions to the original user who logged in.

uid

The user ID of the user who started the process. In this case, 0 for root.

gid

The group ID of the user who started the process. In this case, 0 for root.

euid, suid, fsuid

Effective user ID, set user ID, and file system user ID of the user that started the process.

egid, sgid, fsgid

Effective group ID, set group ID, and file system group ID of the user that started the process.

tty

The terminal from which the application is started. In this case, a pseudoterminal used in an SSH session.

comm

The application name under which it appears in the task list.

exe

The resolved pathname to the binary program.

subj

auditd records whether the process is subject to any security context, such as AppArmor. unconstrained, as in this case, means that the process is not confined with AppArmor. If the process had been confined with audit, the binary pathname plus the AppArmor profile mode would have been logged.

key

If you are auditing a large number of directories or files, assign key strings each of these watches. You can use these keys with ausearch to search the logs for events of this type only.

The second message triggered by the example less call does not reveal anything apart from just the current working directory when the less command was executed.

The third message reveals the following (the type and message flags have already been introduced):

item

In this example, item references the a0 argument—a path—that is associated with the original SYSCALL message. Had the original call had more than one path argument (such as a cp or mv command), an additional PATH event would have been logged for the second path argument.

name

Refers to the pathname passed as an argument to the less (or open) call.

inode

Refers to the inode number corresponding to name.

dev

Specifies the device on which the file is stored. In this case, 03:01, which stands for /dev/hda1 or first partition on the first IDE device.

mode

Numerical representation of the file's access permissions. In this case, root has read and write permissions and his group (root) has read access while the entire rest of the world cannot access the file at all.

ouid and ogid

Refer to the UID and GID of the inode itself.

rdev

Not applicable for this example. The rdev entry only applies to block or character devices, not to files.

Example 1-8 highlights the audit events triggered by an incoming SSH connection. Most of the messages are related to the PAM stack and reflect the different stages of the SSH PAM process. Several of the audit messages carry nested PAM messages in them that signify that a particular stage of the PAM process has been reached. Although the PAM messages are logged by audit, audit assigns its own message type to each event:

type=USER_AUTH msg=audit(1175508928.540:4499): user pid=28731 uid=0 
auid=0 msg='PAM: authentication acct=root : exe="/usr/sbin/sshd" 
(hostname=earth.example.com, addr=192.168.0.1, terminal=ssh res=success)'
type=USER_ACCT msg=audit(1175508928.540:4500): user pid=28731 uid=0  
auid=0 msg='PAM: accounting acct=root : exe="/usr/sbin/sshd" 
(hostname=earth.example.com, addr=192.168.0.1, terminal=ssh res=success)'
type=CRED_ACQ msg=audit(1175508928.544:4501): user pid=28729 uid=0  
auid=0 msg='PAM: setcred acct=root : exe="/usr/sbin/sshd" 
(hostname=earth.example.com, addr=192.168.0.1, terminal=/dev/pts/0 res=success)'
type=USER_LOGIN msg=audit(1175508928.544:4502): user pid=28732 uid=0  
auid=0 msg='uid=0: exe="/usr/sbin/sshd" (hostname=earth.example.com, 
addr=192.168.0.1, terminal=/dev/pts/0 res=success)'
type=USER_START msg=audit(1175508928.548:4503): user pid=28732 uid=0  
auid=0 msg='PAM: session open acct=root : exe="/usr/sbin/sshd" 
(hostname=earth.example.com, addr=192.168.0.1, terminal=/dev/pts/0 res=success)'
type=CRED_REFR msg=audit(1175508928.548:4504): user pid=28732 uid=0  
auid=0 msg='PAM: setcred acct=root : exe="/usr/sbin/sshd" 
(hostname=earth.example.com, addr=192.168.0.1, terminal=/dev/pts/0 res=success)'
    

1.5.2 Generating Custom Audit Reports

The raw audit reports stored in the /var/log/audit directory tend to become very bulky and hard to understand. To find individual events of interest, you might have to read through thousands of other events before you spot the one that you want. To avoid this, use the aureport utility and create custom reports.

The following use cases highlight just a few of the possible report types that you can generate with aureport:

Read Audit Logs from Another File

When the audit logs have moved to another machine or when you want to analyze the logs of a number of machines on your local machine without wanting to connect to each of these individually, move the logs to a local file and have aureport analyze them locally:

aureport -if myfile

Summary Report
======================
Range of time: 04/19/2007 13:42:43.280 - 04/23/2007 21:11:21.533
Number of changes in configuration: 55
Number of changes to accounts, groups, or roles: 0
Number of logins: 20
Number of failed logins: 10
Number of users: 3
Number of terminals: 11
Number of host names: 5
Number of executables: 12
Number of files: 3
Number of AVC denials: 0
Number of MAC events: 0
Number of failed syscalls: 4
Number of anomaly events: 0
Number of responses to anomaly events: 0
Number of crypto events: 0
Number of process IDs: 544
Number of events: 2795

The above command, aureport without any arguments, provides just the standard general summary report generated from the logs contained in myfile. To create more detailed reports, combine the -if option with any of the options below. For example, generate a login report that is limited to a certain time frame:

aureport -l -ts 12:00 -te 13:00 -if myfile

Login Report
============================================
# date time auid host term exe success event
============================================
1. 04/23/2007 12:38:38 PM root earth /dev/pts/0 /usr/sbin/sshd yes 1624

Convert Numeric Entities to Text

Some information, such as user IDs, are printed in numeric form. To convert these into a human-readable text format, add the -i option to your aureport command.

Create a Rough Summary Report

If you are just interested in the current audit statistics (events, logins, processes, etc.), run aureport without any other option:

aureport

Summary Report
======================
Range of time: 04/19/2007 13:42:43.280 - 04/23/2007 21:11:21.533
Number of changes in configuration: 55
Number of changes to accounts, groups, or roles: 0
Number of logins: 20
Number of failed logins: 10
Number of users: 3
Number of terminals: 11
Number of host names: 5
Number of executables: 12
Number of files: 3
Number of AVC denials: 0
Number of MAC events: 0
Number of failed syscalls: 4
Number of anomaly events: 0
Number of responses to anomaly events: 0
Number of crypto events: 0
Number of process IDs: 544
Number of events: 2795

Create a Summary Report of Failed Events

If you want to break down the overall statistics of plain aureport to the statistics of failed events, use aureport --failed:

aureport --failed

Failed Summary Report
======================
Range of time: 04/19/2007 13:42:43.280 - 04/23/2007 21:25:38.406
Number of changes in configuration: 0
Number of changes to accounts, groups, or roles: 0
Number of logins: 0
Number of failed logins: 10
Number of users: 1
Number of terminals: 6
Number of host names: 4
Number of executables: 4
Number of files: 1
Number of AVC denials: 0
Number of MAC events: 0
Number of failed syscalls: 4
Number of anomaly events: 0
Number of responses to anomaly events: 0
Number of crypto events: 0
Number of process IDs: 21
Number of events: 32

Create a Summary Report of Successful Events

If you want to break down the overall statistics of a plain aureport to the statistics of successful events, use aureport --success:

aureport --success

Success Summary Report
======================
Range of time: 04/19/2007 13:42:43.280 - 04/23/2007 21:31:35.865
Number of changes in configuration: 55
Number of changes to accounts, groups, or roles: 0
Number of logins: 20
Number of failed logins: 0
Number of users: 3
Number of terminals: 10
Number of host names: 5
Number of executables: 12
Number of files: 4
Number of AVC denials: 0
Number of MAC events: 0
Number of failed syscalls: 0
Number of anomaly events: 0
Number of responses to anomaly events: 0
Number of crypto events: 0
Number of process IDs: 535
Number of events: 2787

Create Summary Reports

In addition to the dedicated summary reports (main summary and failed and success summary), use the --summary option with most of the other options to create summary reports for a particular area of interest only. Not all reports support this option, however. This example creates a summary report for user login events:

aureport -u --summary

User Summary Report
===========================
total  auid
===========================
5640  root
13  tux
3  geeko
3  news

Create a Report of Events

To get an overview of the events logged by audit, use the aureport -e command. This command generates a numbered list of all events including date, time, event number, event type, and audit ID.

aureport -e

Event Report
===========================
# date time event type auid
===========================
1. 04/23/2007 08:00:01 AM 1507 USER_ACCT unset
2. 04/23/2007 08:00:01 AM 1508 CRED_ACQ unset
3. 04/23/2007 08:00:01 AM 1509 LOGIN root
4. 04/23/2007 08:00:01 AM 1510 USER_START root

Create a Report from All Process Events

To analyze the log from a process's point of view, use the aureport -p command. This command generates a numbered list of all process events including date, time, process ID, name of the executable, system call, audit ID, and event number.

aureport -p

Process ID Report
======================================
# date time pid exe syscall auid event
======================================
1. 04/23/2007 08:00:01 PM 13097 /usr/sbin/cron 0 unset 1888
2. 04/23/2007 08:00:01 PM 13097 /usr/sbin/cron 0 unset 1889
3. 04/23/2007 08:00:01 PM 13097 ? 0 root 1890

Create a Report from All System Call Events

To analyze the audit log from a system call's point of view, use the aureport -s command. This command generates a numbered list of all system call events including date, time, number of the system call, process ID, name of the command that used this call, audit ID, and event number.

aureport -s

Syscall Report
=======================================
# date time syscall pid comm auid event
=======================================
1. 04/23/2007 08:04:08 PM 5 13374 file root 1900
2. 04/23/2007 08:04:08 PM 5 13376 file root 1901
3. 04/23/2007 08:04:08 PM 5 13368 less root 1902

Create a Report from All Executable Events

To analyze the audit log from an executable's point of view, use the aureport -x command. This command generates a numbered list of all executable events including date, time, name of the executable, the terminal it is run in, the host executing it, the audit ID, and event number.

aureport -x

Executable Report
====================================
# date time exe term host auid event
====================================
1. 04/23/2007 08:00:01 PM /usr/sbin/cron cron ? unset 1888
2. 04/23/2007 08:00:01 PM /usr/sbin/cron cron ? unset 1889
3. 04/23/2007 08:00:01 PM /usr/sbin/cron cron ? root 1891

Create a Report about Files

To generate a report from the audit log that focuses on file access, use the aureport -f command. This command generates a numbered list of all file-related events including date, time, name of the accessed file, number of the system call accessing it, success or failure of the command, the executable accessing the file, audit ID, and event number.

aureport -f

File Report
===============================================
# date time file syscall success exe auid event
===============================================
1. 04/23/2007 06:16:38 PM /var/log/audit/audit.log 5 yes /usr/bin/file root 1822
2. 04/23/2007 06:16:38 PM /var/log/audit/audit.log 5 yes /usr/bin/file root 1823
3. 04/23/2007 06:16:38 PM /var/log/audit/audit.log 5 yes /usr/bin/less root 1824

Create a Report about Users

To generate a report from the audit log that illustrates which users are running what executables on your system, use the aureport -u command. This command generates a numbered list of all user-related events including date, time, audit ID, terminal used, host, name of the executable, and an event ID.

aureport -u

User ID Report
====================================
# date time auid term host exe event
====================================
1. 04/23/2007 08:00:01 PM unset cron ? /usr/sbin/cron 1888
2. 04/23/2007 08:00:01 PM unset cron ? /usr/sbin/cron 1889
3. 04/23/2007 08:00:01 PM root ? ? ? 1890
4. 04/23/2007 08:00:01 PM root cron ? /usr/sbin/cron 1891
5. 04/23/2007 08:00:01 PM root cron ? /usr/sbin/cron 1892
6. 04/23/2007 08:00:01 PM root cron ? /usr/sbin/cron 1893
7. 04/23/2007 08:04:01 PM unset ssh 192.168.0.20 /usr/sbin/sshd 1894

Create a Report about Logins

To create a report that focuses on the login attempts to your machine, run the aureport -l command. This command generates a numbered list of all login-related events including date, time, audit ID, host and terminal used, name of the executable, success or failure of the attempt, and an event ID.

aureport -l

Login Report
============================================
# date time auid host term exe success event
============================================
1. 04/23/2007 12:38:38 PM root earth.example.com /dev/pts/0 /usr/sbin/sshd yes 1624
2. 04/23/2007 01:38:12 PM root earth.example.com /dev/pts/1 /usr/sbin/sshd yes 1655
3. 04/23/2007 03:32:58 PM root 192.168.0.20 /dev/pts/0 /usr/sbin/sshd yes 1712

Limit a Report to a Certain Time Frame

To analyze the logs for a particular time frame, such as only the working hours of April 23, 2007, first find out whether this data is contained in the the current audit.log or whether the logs have been rotated in by running aureport -t:

aureport -t

Log Time Range Report
=====================
/var/log/audit/audit.log: 04/19/2007 13:42:43.280 - 04/23/2007 22:19:08.087

The current audit.log contains all the desired data. Otherwise, use the -if option to point the aureport commands to the log file that contains the needed data.

Then, specify the start date and time and the end date and time of the desired time frame and combine it with the report option needed. This example focuses on login attempts:

aureport -ts 04/23/2007 8:00 -te 04/23/2007 17:00 -l

Login Report
============================================
# date time auid host term exe success event
============================================
1. 04/23/2007 12:38:38 PM root earth.example.com /dev/pts/0 /usr/sbin/sshd yes 1624
2. 04/23/2007 01:38:12 PM root earth.example.com /dev/pts/1 /usr/sbin/sshd yes 1655
3. 04/23/2007 03:32:58 PM root sun.example.com /dev/pts/0 /usr/sbin/sshd yes 1712

The start date and time are specified with the -ts option. Any event that has a time stamp equal to or after your given start time appears in the report. If you omit the date, aureport assumes that you meant today. If you omit the time, it assumes that the start time should be midnight of the date specified. Use the 24 clock notation rather than the 12 hour one and adjust the date format to your locale (specified in /etc/sysconfig/audit under AUDITD_LANG, default is en_US).

Specify the end date and time with the -te option. Any event that has a time stamp equal to or before your given event time appears in the report. If you omit the date, aureport assumes that you meant today. If you omit the time, it assumes that the end time should be now. Use a similar format for the date and time as for -ts.

All reports except the summary ones are printed in column format and sent to stdout, which means that this data can be piped to other commands very easily. The visualization scripts introduced in Section 1.8, Visualizing Audit Data are just one example of how to further process the data generated by audit.