Mail Handling under Linux

Matthias Friedrich

Introduction

When running under Windows, mails are usually received and sent via the mail client.

Although this method can also be used for Linux with programs such as Netscape Communicator, it is more common to apply a different procedure allowing greater ease of use and multi-user operation. Independently of the features of a given mail reader, mails can be emptied from several POP3 accounts simultaneously and delivered to local users. IMAP will not be considered in this article.

The process is, however, a little more complicated because several programs are involved:

A mail is fetched from the POP3 account by fetchmail and forwarded to the local mail server (sendmail, postfix or similar). This then transfers the mail to procmail, which carries out a few filter functions and then saves the mail to a local mailbox according to pre-configured rules. The mail can then be read by a mail program such as mutt.

The mail program transfers an outgoing mail to the local server, which forwards it to the SMTP server of the Internet provider when an Internet connection is established.

Before you can begin configuration, you need to make a few specifications regarding your own mail account and the mail accounts for the other users (if applicable). See the following example:

 
 +---------------+--------------------------+------------------------------+
 | Mail address: |  skywalker@tatooine.net  |  princess.leia@alderaan.net  |
 +---------------+--------------------------+------------------------------+
 | Account:      |  skywalker               |  2345123                     |
 | Password:     |  yoda                    |  obi-wan                     |
 | POP3 server:  |  pop.tatooine.net        |  pobox.alderaan.net          |
 | SMTP server:  |  mail.tatooine.net       |  mailout.alderaan.net        |
 +---------------+--------------------------+------------------------------+
 | Local users:  |  luke                    |  leia                        |
 +---------------+--------------------------+------------------------------+

Inbound mail

fetchmail

The first step is the collection of the mail from a POP3 server. This is where fetchmail, an extremely powerful and well-documented program, is used. At first, however, you only need to understand a few of its capabilities.

In the configuration file /etc/fetchmailrc, enter the following (adapted to your own data, of course):

 # Fetch mail for Luke
 poll pop.tatooine.net protocol pop3 user "skywalker" there
 with password "yoda" is luke here

 # Fetch mail for Leia
 poll pobox.alderaan.net protocol pop3 user "2345123" there
 with password "obi-wan" is leia here

The first entry triggers the collection of all mails for the "skywalker" account from POP3 server "pop.tatooine.net". The password is "yoda" and the mail is intended for the local user "luke".

The second entry works in exactly the same way. You can also see that the account name on the POP3 server does not always agree with the local part of the e-mail address.

Because the configuration file contains passwords, it must be protected from prying eyes:

 # chmod 600 /etc/fetchmailrc

When you enter

 # fetchmail -f /etc/fetchmailrc

new mails are collected and saved locally in the default mailbox /var/mail/user (or in older systems in /var/spool/mail/user).

Until everything is functioning as required, it makes sense to call fetchmail with the option -k, ensuring that downloaded mails remain on the server. The option -v provides detailed (verbose) diagnostic information for fetchmail.

You can find a full description of the command line options and the configuration file in the manpage (man fetchmail).

procmail

Procmail ("process mail") is used to continue processing of mails. Its functions include sorting of mail into different folders according to sender. It can also delete spam and forward mails to other programs. Skip this section if you do not wish to use a mail filter.

To ensure that procmail receives the mail that you want it to sort, your mail server (sendmail, exim or similar) must be configured so that it delivers local mails via procmail (this is the case for most solutions).

If, for any reason, you cannot get your mail server to work in conjunction with procmail, try creating a ~/.forward file (described in the procmail manpage, NOTES section).

When you have ensured that procmail is receiving incoming mails, you can start to write filter rules for the mail. In this example we want to save incoming mails in the directory ~/Mail (MAILDIR) to different mail folders.

The file ~/.procmailrc contains a series of filter rules that are processed sequentially until a rule applies to the mail. If no hits are found, then procmail saves the mail to the default folder ("Inbox").

Contents of ~/.procmailrc:

 # This directory contains the following mail folders
 MAILDIR=$HOME/Mail

 # The "log" file is where procmail lists where the individual mails
 # are saved. This is especially useful for test purposes.
 LOGFILE=$MAILDIR/log

 # Mails from admiralackbar@rebelhqs.org are placed in the "important" folder.
 :0:
 * ^From:.*admiralackbar@rebelshq.org
 important

 # Unwanted mails are placed in "unimportant", a staging point
 # on the way to /dev/null
 :0:
 * ^From:.*ewoks@endor.net
 unimportant

 # The mails from mailing list discussion@jedi-today.org are placed in the
 # jedi-list folder
 :0:
 * ^TO_infos@jedi-today.org
 jedi-list

 # This is then forwarded directly to vader@galactic-empire.org
 :0:
 * ^TO_newsletter@sith-magazine.com
 ! vader@galactic-empire.org

Sometimes mailing lists are not so easy to filter, as in the example. In this case you need to search for the header that contains the list address. If, for example, the list header is "Sender: infos@jedi-today.org", then the rule is as follows:

 :0:
 * ^Sender: infos@jedi-today.org
 jedi-list

When you take a closer look at the procmail documentation, you will recognize that the filter rules are regular expressions (only, of course, if you know what these are ;-)). You will also see that "infos@jedi-today.org" could be written as "infos@jedi-today\.org", and that mails are very well suited for the entry of your own scripts ;-)

If you already have a mailbox in mbox format, as used in Netscape, for example, then you can use the command

 $ formail -s procmail < old_mailbox

to sort your mailbox retroactively using procmail. This is also a good test for the new procmail filter.

The procmailex manpage (man procmailex) provides a great deal of useful examples. man procmailrc explains in detail the setup of the configuration file and man procmail provides information on command line arguments and diagnostic messages, among others.

You can find a more detailed introduction to the subject in [1]. You can find full details on procmail, including scoring and complex processing techniques in [2]. [3] provides a few scripts and links for dealing with spam.

Mail clients

After the mail has been sorted, it can be read using a mail client.

We recommend the console program mutt, which provides a wide range of functions and which can be closely configured according to your requirements. To begin with, however, you only need to know how to carry out a basic configuration of ~/.muttrc. This is as follows:

 # this is where your mail is saved
 set folder = ~/Mail

 # read mails from /var/mail/<user> are placed in ~/Mail/mbox
 set mbox = +mbox

 # new mails could be placed here
 mailboxes +important +unimportant +jedi-list

 # mails that have been sent are placed in ~/Mail/sent-mails
 set record = +sent-mails

 # you can enter your preferred editor here
 set editor = "/usr/bin/vi"

mutt automatically replaces the plus sign with the contents of the folder variable. In this case, +sent-mails stands for ~/Mail/sent-mails. Within mutt, you can enter c to switch to another mailbox.

The mutt package also includes an excellent handbook, in which you should read at least the "Getting Started" chapter. You can find the most recent version in the official mutt homepage under [4].

Outbound mail

When sending mails, a local mail server such as sendmail or postfix is used. The mail client forwards mails to the mail server, which then attempts to deliver the messages. If this is not currently possible because there is no connection to the Internet, then the mails are placed in a queue. You can, of course, use the new SUSE eMail Server II, which administers all mails according to the IMAP standard. A description of this server is not, however, within the scope of this article.

Because mail servers are not easy to configure, and every solution uses its own tools for this purpose (YaST for SUSE, eximconfig for Debian), a full description would be beyond the scope of this article. If you would rather experiment a little for yourself, you can find the sendmail manual in the sendmail homepage sendmail in [5].

For a basic configuration it is normally sufficient to specify a "Smarthost". This is an SMTP server of the provider, which receives and delivers incoming mails. We recommend that you do not have the queue processed automatically at all times, because most of the time there will be no Internet connection.

If the sender address (in From) takes the form <username@localhost>, then you need to have the mail server convert it automatically into a valid address.

In sendmail an entry must be made in /etc/mail/genericstable for every mail address:

 luke skywalker@tatooine.net
 leia princess.leia@alderaan.net

Then, another

 # makemap hash /etc/mail/genericstable < /etc/mail/genericstable

must be executed for the entry to take effect.

In exim, entries are appended at the end of /etc/exim.conf:

 luke@localhost skywalker@tatooine.net   Ff
 leia@localhost princess.leia@alderaan.net   Ff

When an Internet connection is established, you can enter

 # sendmail -qf

to deliver the messages in the queue. This command also works with exim, smail or postfix.

Automation

The script /etc/ppp/ip-up is processed automatically every time a connection is established. At the end of this file you can append the commands for fetching and sending your mail:

 # Fetch mail
 fetchmail -f /etc/fetchmailrc --invisible --syslog

 # Send mail
 sendmail -qf

The sequence is important, because freemail providers in particular, often require SMTP-after-POP. In this case it is only possible to send mails if you have first collected mail via POP3.

The --invisible option when calling fetchmail ensures that the local server "thinks" that the mails have been sent directly from another server without coming via a POP3 mailbox.

The --syslog option writes information to /var/log/messages, which can be useful if problems occur.

If /etc/ppp/ip-up does not yet exist, you can recreate the file. In this case, remember that the first line must be

 #! /bin/sh

and that the script with command

 # chmod 755 /etc/ppp/ip-up

must be marked as executable.

You should now be able to use mail in conjunction with Linux.

Links

[1] http://www.ii.com/internet/robots/procmail/qs/
[2] http://pm-doc.sourceforge.net/pm-tips.html
[3] http://www.telebyte.com/stopspam/
[4]http://www.mutt.org/doc/manual/
[5] http://www.sendmail.org/~ca/email/doc8.10/op.html

Many thanks to the members of the dcouli-authors list, who helped me to write this article:

• Roland Rosenfeld
• Jo Moskalewski
• Karl-Heinz Zimmer
• Thomas Bader
• Benjamin Greiner