Linux tail command

The Linux tail command is an essential tool for the command line. The command is primarily used to output theend of a (text) file or to limit the output of a Linux command. The Linux tail command is thus in line with the Linux head command and “cat” and “less” commands. These Linux commands are used to output the contents of text files.

The Linux tail command is part of the GNU Core Utilities (Coreutils) – a collection of basic commands, contained in the open-source operating system Linux. The Coreutils are published under an open-source license and are available for download for many different operating systems.

The basic functionality of the Linux tail command is to output the end of a file. Typically, new data added to a file ends up at its tail (i.e., the end). So, the Linux tail command allows us to check if a file has new data attached. Therefore, the Linux tail command is a popular tool to evaluate and monitor log files.

Many programs, especially web servers like Apache or nginx write status information into so-called log files. For example, each line in a server log file contains a time stamp, the URL of the requested resource and the IP address of the requesting party.

But log files can easily grow to a huge size with every single request that is made. To limit the expansion of file sizes, log files are usually “rotated”. The log file is compressed and archived under a new name. Furthermore, a new, empty log file is created under the original name. Here is an overview of common log files under Ubuntu Linux:

You can call up the Linux tail command from the command line. As usual, the name of the command is entered and followed by optional parameters. The call is completed by the name or path of one or more files. Let’s look at a general use case:

Without options, the simplest call of the Linux tail command results in the following pattern:

Executed this way, the Linux tail command prints the last ten lines of the specified file. This is useful when you want to view the final data written to the file.

The Linux tail command can be controlled using parameters. As part of the GNU Coreutils, each option has a long-form and more frequently used options are given a short-form. Here is an overview of the most useful option parameters:

The basic options of the Linux tail command shown are functionally analogous to those of the head command. However, while the head command returns the beginning of a file, the Linux tail command outputs the end of a file. But the command is capable of so much more.

In particular, the Linux tail command provides plenty of options to monitor files for changes. With the Linux tail command, data that has been added to the end of the file can be output continuously. The command is therefore particularly suitable for monitoring log files. This process is also known as “live tail”. Here is an overview of the options that are most frequently used:

The Coreutils documentation lists the Linux tail command in the “Output of parts of files” section. You should know that the term “files” is used in a broad sense. The term “text streams” is more precise.

Following the Unix philosophy, command line commands use text streams as a universal input and output format. Text streams are mainly files but include the standard input and output of the command line. Further, so-called “pipes” are very important because they allow for several commands to be linked. The output of a command is passed on as an input to the next command.

The underlying idea of chaining several commands together harks back to Unix philosophy. Instead of developing complex commands for completely different tasks, there is a relatively manageable set of general commands. Following the saying “do one thing, and do it well”, each command has a narrowly defined functionality. The commands use text streams as a universal interface and can be combined to create new solutions repeatedly.

The following examples are based on the basic options of the Linux tail command shown above. For examples of advanced options, please see the section on monitoring log files below.

Most of the examples shown combine the Linux tail command with one or more Linux commands. The pipes mentioned are used to output a command as an input for the next command.

In the simplest case, we use the Linux tail command to output the final lines of a file. To try this, replace the placeholder “<file>” with the name or path of an existing text file on your system. Here we use the -n option and output the last three lines of a file:

It can be useful to keep track of the context of the file being processed by the Linux tail command. To output the lines with line numbers, we first process the file with the nl command (the name comes from “line numbering”) and pipe the output to the Linux tail command:

The history command shows commands entered to the command line. Often, the final entries are what most people are after. Further, the output of the command history may reveal sensitive data. In this example, we are forwarding the output of the history command to the Linux tail command and outputting the final five lines:

Let’s look at a more complex example. We will delete the ten oldest files from a directory with backup files. To do this, we use the four Linux commands “ls”, “tail”, “xargs” and “rm”:

What is happening here?

1. The ls command lists the files in a directory.

Using the “-t” option, the files are sorted according to their modification date. The oldest files are at the bottom of the list.

We use the search pattern “* .bak” for the data source. This is a common file extension for backup files. This also serves as a protection so that you don’t accidentally delete important files when you open it.

We pipe the file list to the Linux tail command.

2. Using the Linux tail command without specifying options, we will read the last ten files from the list. The output filenames are the oldest files in the directory.
3. The xargs command takes a list of files and the name of a command. The command is executed when the files are passed as arguments.

In our example, the Linux tail command returns text with several lines. Each line contains the name of a file to be deleted. The file names are separated from the lines via “xargs” and passed as arguments to the rm command.

4. To delete files in Linux, we use the rm command which deletes the file that has been passed as arguments.

Note that deleted files will not be moved to the recycling bin but will be deleted immediately.

Beware when deleting files using the Linux command line. To try the example below, it’s best to first create a directory of test files. To do this, run the following code on the command line:

First, we create a test directory “tail-test/” on the desktop and switch to the directory. Then we create 100 empty test files with “test-1.bak” to “test-100.bak”. Finally, we run the code to delete the ten oldest files:

So far, we’ve considered the general functionality of the Linux tail command. In the following examples, we’ll focus on live monitoring of server log files (“live tail”).

In these examples, we are referring to the Apache2 web server under Ubuntu Linux where log files are placed in the “/var/log/apache2/” directory. The storage location may differ across other Linux distributions. The same applies to other web servers, such as nginx.

A normal call to the Linux tail command outputs the specified number of lines in a file once. If new data is appended to the end of the file, the command must be executed again. There are additional options for this case. These instruct the command to monitor a file for changes and to output them continuously.

Let’s take a look at how this works using the example of the access log of the Apache2 web server. We call the Linux tail command with the -f option and pass on the name of the log file:

Executed this way, changes to the access log are continuously output. This approach is impractical if there are many accesses per unit of time. In this case, the log changes so quickly that the terminal is flooded with data. Press the key combination “Ctrl + C” to cancel a current execution of the live tail.

The Linux tail command monitors the specified file once executed with the ‘-f’ option. If this is removed, the output is canceled. As explained at the beginning, log files are rotated periodically. A new file is created under the previous name. To continue monitoring a log file despite the rotation, we use the -F option:

So far, we’ve processed a single file using the Linux tail command. However, the command allows multiple files to be monitored at the same time. Below we use the search pattern “* .log” to continuously monitor all files with a “.log” extension:

There are two other options for processing multiple files. On the one hand, we can suppress the output of the file names with the “-q” option:

Similarly, the -v option can be used to force the output of the file names. This makes sense if the search pattern only returns a single file:

In previous examples, we’ve monitored a log file for any change. However, it is more common to restrict monitoring to only select changes. To limit the output of changes to a log file to a search pattern, we use the grep command. Here is a general example of this approach:

It makes sense to monitor a server log for access using a specific IP address. This can be useful, for example, to determine whether an attack on a server has already been completed or is still in progress. Here we monitor the access log of the Apache web server for requests from the IP address “93 .184.216.34”:

Let’s look at another common deployment scenario. Instead of filtering the access log by IP address, we monitor access to a specific resource. Access to the “robots.txt” file indicates requests from search engine bots. Again, we make use of the grep command. This way only new accesses from search engine bots are displayed: