Mail Filtering with Procmail

This document is meant as a gentle introduction to the use of Procmail. More detailed information can be found on the Procmail homepage www.procmail.org

What is it?

 Procmail is a program for filtering electronic mail. It is very useful for presorting and preprocessing large amounts of incoming mail. You can use it to sort out mail from mailing lists, to dispose of junk mail, to send automatic replies, or even to run a mailing list.

You control Procmail yourself, through a file that you put in your home directory.

Important note: If you are using some other mechanism to deliver your mail (if you have a .forward file with anything other than procmail in it) - your .procmailrc will be ignored and procmail filtering will not work.
Side Note -- a bit of Unix trickery

Files in Unix that begin with a dot '.' are hidden files. So, when you use the ls command to view the files in your home directory, you may not see the .procmailrc file, or any other so-called "dot-files", right away. To see hidden files in your directory, use the '-a' option, as in ls -a. The '-a' stands for "all files," and will show you both hidden and visible files in one listing.

The whole trick to Procmail is writing the .procmailrc file.

The .procmailrc file filters mail based on the Sophos spam rating system. All the mail that comes into the cs.columbia.edu domain through our mail gateways, is process by Sophos Puremessage. All virus mail is deleted, virus attachments are removed, and the remaining mail is marked with at least two lines similar to the following:

X-PMX-Version: 4.7.0.111621, Antispam-Engine: 2.0.2.0, Antispam-Data: 2004.11.25.56
X-PerlMx-Spam: Gauge=XXXXXXXXXIIIIII, Probability=96%, Report='URI_CLASS_FINANCIAL_DOMAIN 8, RCVD_IN_CBL 3, CTYPE_JUST_HTML 0.848, __CT 0, __CTE 0, __CTYPE_HTML 0, __CTYPE_IS_HTML 0, __HAS_MSGID 0, __MIME_HTML 0, __MIME_HTML_ONLY 0, __MIME_VERSION 0, __SANE_MSGID 0, __TAG_EXISTS_HTML 0'

Procmail can be used to search for these lines and sort out the spam at whatever level you want. You can filter using the 'Gauge', the 'Probability', or a specific Sophos classification type and value found in the 'Report' field.

[ Download ] 
#########################################################################
#                                                                       #
#               STANDARD .PROCMAILRC TO FILTER SPAM                     #
#                                                                       #
#               COMPUTER SCIENCE DEPARTMENT                             #
#                   COLUMBIA UNIVERSITY                                 #
#                                                                       #
#########################################################################

#### Customize the configuration to fit your need. *********
#
## Make sure that this file is only writable by the owner, otherwise
## procmail will not use it (considered "Suspicious" by procmail)
## and complain (silently to you) in the mail log file of the delivering
## machine.
#

# All folders (spam, log, procmailbackup) are in $MAILDIR/mail
MAILDIR=$HOME/mail

# Please empty the mail log ($MAILDIR/log) frequently because it gets very 
# big quickly. This file is a good place to check your filtered email to 
# make sure they are all spam.
LOGFILE=$MAILDIR/log
VERBOSE=off


##### uncomment below to keep backups of all incoming mail
#
# Uncomment the following 3 lines if you want to backup your mail 
# BEFORE filtering, please empty the folder frequently because it gets very 
# big, very quickly.

#BACKUPSPOOL=$MAILDIR/procmailbackup
#:0 c:
#$BACKUPSPOOL

# If email is marked with 90% or higher probability of being spam, filter it.
:0:
* ^X-PerlMx-Spam: Gauge=XXXXXXXXX

# Put the spam in $MAILDIR/spam.
# Please empty the spam folder frequently.
spam

##### or you can delete it
#/dev/null

############################## E N D ######################################
For more aggressive filtering, the following can be used, however, one should be careful if you receive mail in what Sophos classifies as "CHARSET_FARAWAY", e.g. Chinese or Cyrillic.

[ Download ] 
#### Customize the configuration to fit your need. *********
#
## Make sure that this file is only writable by the owner, otherwise
## procmail will not use it (considered "Suspicious" by procmail)
## and complain (silently to you) in the mail log file of the delivering
## machine.
#

# All folders (spam, log, procmailbackup) are in $MAILDIR/mail
MAILDIR=$HOME/mail

# Please empty the mail log ($MAILDIR/log) frequently because it gets very 
# big quickly. This file is a good place to check your filtered email to 
# make sure they are all spam.
LOGFILE=$MAILDIR/log
VERBOSE=off


##### uncomment below to keep backups of all incoming mail
#
# Uncomment the following 3 lines if you want to backup your mail 
# BEFORE filtering, please empty the folder frequently because it gets very 
# big, very quickly.

#BACKUPSPOOL=$MAILDIR/procmailbackup
#:0 c:
#$BACKUPSPOOL

# If email is marked with 60% or higher probability of being spam, filter it.
:0
* ^X-PerlMx-Spam: Gauge=XXXXXX.*
spam

# These are Sophos spam classification strings which are currently useful.

# Buy your drugs from Upper Slobovia (no offense to slobovians intended)
:0
* ^X-PerlMx-Spam: .*GENERAL_MEDS_DRUGS.*
spam

# Chinese spam, and others
:0
* ^X-PerlMx-Spam: .*CHARSET_FARAWAY.*
spam

# If you were using this, you might want to comment it out. #
# It seems to pick up too much real mail.                   #
#                                                           #
# ...I have this account with $1M, but I need you to lend me money to get it.
# :0
# * ^X-PerlMx-Spam: .*FRAUD_419.*
# spam

# html spam that uses graphics to get around text analysis
:0
* ^X-PerlMx-Spam: .*IMGSPAM.*
spam

############################## E N D ######################################
The following example is for users who do not currently use any filtering software.

Sample .procmailrc: 

# .procmailrc
# routes incoming mail to appropriate mailboxes
PATH=/usr/bin:/usr/local/bin
MAILDIR=$HOME/mail   # all mailboxes are in mail - Pine looks there by default
LOGFILE=.procmail_log
SHELL=/bin/sh

# Put mail from foo into folder foo
:0:
* ^(From|Cc|To).*foo
foo

#all other mail goes to DEFAULT mailbox

Download
The above is example is a valid .procmailrc file and you can just download and start using it, modifying it for your needs. (Rename it to .procmailrc in your home directory after download.).

Note: If you are using pop to read your mail, your mailer will not be able to access anything other than your inbox; so please use imap or local mail readers.

If you use mh-style mail, and use .maildelivery for filtering your mail, you can use .procmailrc to do the same thing: Sample .procmailrc for use with mh-style mail: Download 

# .procmailrc
# routes incoming mail to appropriate mailboxes
PATH=/usr/bin:/usr/local/bin:/usr/local/mh/bin:/usr/lib/nmh
LOGFILE=.procmail_log
SHELL=/bin/sh

# Put mail from foo into folder foo
:0:
* ^(From|Cc|To).*foo
|rcvstore foo

#default - should be last recipe
:0:
|rcvstore inbox