mailprocessing reference

mailprocessing consists of two programs: imapproc and maildirproc. Both run a user-defined piece of Python code to filter email.

# imapproc

imapproc continuously monitors one or more remote IMAP folders and processes mail in them according to user defined Python code in an rc file, a user defined piece of Python code. It also logs its actions to a log file. When there is no more mail to process, imaproc sleeps one second and then checks again. And so on. To make imapproc exit after the first filtering run, pass the --once option.

imapproc keeps a list of folders in the IMAP account to process. At least least one IMAP folder must be specified for imapproc to run. Folder paths are always absolute. There are two ways to specify this information: by passing command line options, or by setting attributes on the processor object in the rc file. The rc file has priority over the command line options.

The typical folder to run imapproc on is `INBOX` but you can of course specify multiple folders to process.

The default location of the rc file for imapproc is `~/.mailprocessing/imap.rc` and the default location of its log file is `~/.mailprocessing/log-imap`.

imapproc can optionally reload the rc file whenever a modification is detected (that is, the file's mtime has changed). This automatic reloading is turned off by default and can be enabled either by passing the `--auto-reload-rcfile` command line option or by setting the `auto_reload_rcfile` property to `True` on the processor object in the rc file.

imapproc writes its process ID to `~/.mailprocessing/imapproc.pid` and uses that PID file for locking as well. If you want to run multiple imapproc instances in parallel, use the `--pidfile` and `--logfile` options to give each process a different PID and log file.

# maildirproc

maildirproc continuously monitors one or more maildirs and processes mail in them according to logic in an rc file, a user defined piece of Python code. It also logs its actions to a log file. By default maildirproc operates continuously: when there is no more mail to process, maildirproc sleeps one second and then checks again. And so on. To make maildirproc exit after the first filtering run, pass the --once option.

maildirproc keeps a list of maildir directories to process. At least one maildir directory must be specified for maildirproc to run. A maildir directory path can be absolute (starting with a slash) or non-absolute. In the latter case, it will be considered relative to the maildir base directory, which defaults to the current working directory. There are two ways to specify this information: by passing command line options, or by setting attributes on the processor object in the rc file. The rc file has priority over the command line options.

In a Maildir++-style setup, the maildir base directory should typically be set to ~/Maildir and the maildir list should include the directory . to make maildirproc process the inbox.

The default location of the rc file for maildirproc is `~/.mailprocessing/maildir.rc` and the default location of its log file is `~/.mailprocessing/log-maildir`.

imapproc can optionally reload the rc file whenever a modification is detected (that is, the file's mtime has changed). This automatic reloading is turned off by default and can be enabled either by passing the `--auto-reload-rcfile` command line option or by setting the `auto_reload_rcfile` property to `True` on the processor object in the rc file.

# Command line options

## Common options

maildirproc and imapproc both take the following common command line options:

--version

  show program's version number and exit

-h, --help

  show this help message and exit

--auto-reload-rcfile

  turn on automatic reloading of the rc file when it has been modified

--dry-run

  just log what should have been done; implies --once

-l FILE, --logfile=FILE

  send log to FILE instead of the default (~/.maildirproc/log-imap for imapproc and ~/.maildirproc/log-maildir for maildirproc). If you are running multiple maildirproc or imapproc instances, you must specify a dedicated log file for each of these.

--log-level=INTEGER

  only include log messages with this log level or lower; defaults to 1

--once

  only process the maildirs once and then exit; without this flag, maildirproc will scan the maildirs continuously

-r FILE, --rcfile=FILE

  use the given rc file instead of the default (~/.maildirproc/default.rc)

--test

  test mode; implies --dry-run, --once, --logfile=- and --verbose

-v, --verbose

  increase log level one step

## maildirproc-specific options

The following options are specific to maildirproc:

-m DIRECTORY, --maildir=DIRECTORY

  add DIRECTORY to the set of maildir directories to process (can be passed multiple times); if DIRECTORY is relative, it will be considered relative to the maildir base directory

-b DIRECTORY, --maildir-base=DIRECTORY

  set maildir base directory; defaults to the current working directory

-p PREFIX, --folder-prefix

  prefix Maildir names with PREFIX; defaults to '.'

-s SEP, --folder-separator=SEP

  use sep as a folder separator in maildir names; defaults to '.'. List style folder names passed to create_folder() will be joined by this character, e.g. ["github", "jrosdahl", "maildirproc"] will become ".github.jrosdahl.maildirproc.

## imapproc-specific options

-C, --cache-headers

  Whether to cache the email headers retrieved from the IMAP server. By default caching is disabled.

--cache-file FILE

  Store the email header cache in FILE if caching is enabled. If this is not specified explicitly, ~/.maildirproc/*HOST*.cache will be used, where HOST is the IMAP server's host name passed set with the --host option.

-c CERT, --certfile

  Use SSL certificate file CERT to verify IMAP server's SSL certificate (only relevant for IMAPS)

-H HOST, --host

  Connect to IMAP server HOST. This option is mandatory

-i INTERVAL, --interval

  Scan IMAP folders for new email every INTERVAL seconds; defaults to 300; will be ignored if --once is specified as well

--flag-batchsize SIZE

  Batch size to use when fetching message flags. Defaults to 200. When there are more messages to fetch flags for, multiple FETCH commands will be issued, each with SIZE or fewer messages. Reduce this value if you experience session expiry during message flag retrieval.

--header-batchsize SIZE

  Batch size to use when fetching message headers. Defaults to 200. When there are more messages to fetch headers for, multiple FETCH commands will be issued, each with SIZE or fewer messages. Reduce this value if you experience session expiry during message header retrieval.

--folder-prefix PREFIX

  Prefix folder names with the string PREFIX. This is relevant for some IMAP servers that store all folders as subfolders of INBOX. imapproc will attempt to detect this situation, but this detection may not work with all server side IMAP implementations. If this is the case for your IMAP server, use this option to specify a prefix explicitly.

--folder-separator SEP

  Use SEP as a separator for folder hierarchies. By default, imapproc will determine the folder separator from the server's LIST response. This should work fine for most IMAP servers.

-p PW, --interval

  Use password PW to authenticate against IMAP server; since this will show up in the process list, this is mainly intended for debugging. Use --password-command otherwise. Either this option or --password-command is mandatory.

--pidfile FILE

  Write the imapproc process' PID to FILE rather than the default location ~/.imapproc/imapproc.pid. This file is also used for locking so if you need to run multiple different imapproc processes in parallel you need to specify different PID and log files for these.

--password-command CMD

  Run command CMD and send use its output as the password to authenticate against the IMAP server with. This is the recommended approach for specifying the IMAP password. Either this option or --password is mandatory.

-P PORT --port

  IMAP port to use. Defaults to 143 if --use-ssl is not specified and 993 if it is.

-s --use-ssl

  Use SSL to connect to the IMAP server (default: no).

-U USER --user

  Log in to the IMAP server with user name USER. This option is mandatory.

--insecure

  Do no certificate validation when connecting to an SSL IMAP server (default: no). This means the certificate subject names will be ignored, as will any certificate authorities. Your connection will not be protected from active attackers.

# Configuration

maildirproc's and imapproc's configuration, the rc file, is not a set of declarative rules. Instead, it is a simple Python program that has access to a "maildir processor" object which produces mail objects. The mail processing logic is defined in terms of if/elif/else statements and actions are performed by calling methods on the mail objects.

Maildir and IMAP specific functionality is implemented by the MaildirProcessor and ImapProcessor classes, respecitively.

## The MailProcessor class

Iteration over a MailProcessor instance yields Mail instances. imapproc and maildirproc will create a MailProcessor instance for you which is available as the global processor variable in the rc file's name space.

### Readable and writable properties

auto_reload_rcfile

  Whether the rc file should be automatically reloaded when it has been modified. Assignment to this property overrides the corresponding command-line option.

maildir_base

  The base directory of maildirs. Assignment to this property overrides the corresponding command-line option. This property is specific to MaildirProcessor instances.

maildirs

  A list of maildirs (subdirectories of the maildir base directory). Assignment to this property overrides the corresponding command-line option. This property is specific to MaildirProcessor instances.

folders

  A list of IMAP folders. Assignment to this property overrides the corresponding command-line option. This property is specific to ImapProcessor instances.

### Methods

create_folder(*folder*, parents=True, prefix='.')

  Create folder folder (a string, or a list of namespace components). This method can safely be called for existing folders. If the folder exists already, the method will log that it exists and exit without trying to create it. For MaildirProcessor folder does not need to be on the same file system as the mail. If the folder path is relative, it will be considered relative to the maildir base directory. The boolean keyword argument parents governs whether parent folders should be created as well, e.g. if you create 'github.jrosdahl.maildirproc', 'github' and 'github.jrosdahl' will be created as well. This is the default behaviour. The prefix keyword argument specifies a prefix to prepend the folder name with. This defaults to the processors prefix attribute which is set via the --prefix command line attribute for MaildirProcessor instances.

### Writable properties

logfile

  Location of the log file. Assignment to this property overrides the corresponding command-line option.

## The Mail class

Indexing a `Mail` instance with a header name (a string) returns a `Header` instance. Example:

    for mail in processor:
      myheader = mail['From']

### Readable properties

folder

  The IMAP folder in which the mail is situated. Only applicable for ImapProcessor.

maildir

  The maildir in which the mail is situated. Only applicable for MaildirProcessor.

path

  Full filesystem path to the mail. Only applicable for MaildirProcessor.

target

  A Target instance.

### Methods

copy(*maildir*, create=False)

  Copy the mail to maildir (a string). maildir does not need to be on the same file system as the mail. If the maildir path is relative, it will be considered relative to the maildir base directory. If the optional create keyword argument is set to True, the folder (and its parent folders) will be created if it does not exist. By default non-existent folders are not created.

delete()

  Delete the mail.

forward(*addresses[, env_sender]*)

  Forward the mail to one or several e-mail addresses and delete the mail. addresses can be either a string or a list of strings. env_sender (optional) specifies which envelope sender address to use.

forward_copy(*addresses[, env_sender]*)

  Forward a copy of the mail to one or several e-mail addresses. addresses can be either a string or a list of strings. env_sender (optional) specifies which envelope sender address to use.

from_mailing_list(*list*)

  Check whether the mail originated from the mailing list list (a string). Currently, the headers Delivered-To, Mailing-List, X-BeenThere and X-Mailing-List are checked. Returns a boolean.

strict_mailing_list(*list*)

  Check whether the mail originated from the mailing list list (a string). It is a bit stricter than `from_mailing_list()` and only matches the content of typical mailing list headers, namely List-Archive, List-Help, List-ID, List-Post, List-Subscribe and X-Mailing-List.

is_seen()

  Returns True if the message has been seen by the user, False otherwise.

is_flagged()

  Returns True if the message has been flagged by the user, False otherwise.

move(*maildir*, create=False)

  Move the mail to maildir (a string). maildir must be on the same file system as the mail, otherwise nothing will happen and an error will be logged. For MaildirProcessor, a relative maildir path, will be considered relative to the maildir base directory. If the optional create keyword argument is set to True, the folder (and its parent folders) will be created if it does not exist. By default non-existent folders are not created.

## The Header class

### Methods

contains(*case-insensitive-string*)

  Check whether case-insensitive-string is part of the header. Returns a boolean.

matches(*case-insensitive-regexp*)

  Check whether case-insensitive-regexp (with an implicit .* prefix) matches the header. Returns a boolean.

## The Target class

### Methods

contains(*case-insensitive-string*)

  Check whether case-insensitive-string is part of the To or Cc header. Returns a boolean.

matches(*case-insensitive-regexp*)

  Check whether case-insensitive-regexp (with an implicit .* prefix) matches the To or Cc header. Returns a boolean.

# Examples

For some mailprocessing configuration examples, see the examples directory. You will also find sample logrotate configuration files for mailprocessing in there.