IntroductionTopContents

Contents

  • Contents
  • Introduction
  • Policy Proposal
  • System-wide Deliverymethod
  • Package "defaultdelivery"
  • Mailhandling package requirements
  • Virtual mailhandling packages (optional)
  • Transition
  • Steps to implementation
  • Different rationale
  • Tecnical details
  • Mail user agents pittfalls
  • One problem with the default configuration
  • Another issue are the different mailbox nameing conventions
  • List of (potential) mailhandling packages
  • Notes
  • Credits
  • Introduction

    As far as I can tell, there exist three popular "User Mailbox" formats: mbox, Maildir and MH, for which a wealth of delivery methods, filter programs and compatible mail user agents are available in Debian.

    The actual policy prescribes the use of the mbox-style /var/mail/<user> delivery method. Any divergent configuration has to be done by hand, and involves non-trivial administration tasks upon upgrade or instalation of any package which accesses the users mailbox.

    Even worse, there are several programs, especially MUA's for the X Window System and mail notification aplications which have to be set up individually by the user, because they are not preconfigured for the standard system inbox, or cannot be made system-wide configurable.

    This leads to situations, where Debian end users are incapable to install a sensitive system to their like, or where administrators spend hours of post-install-configuration, to adapt computers to the End-users needs.

    By letting the administrator make a definitive choice at some point of the installation, and making all respective packages comply with this choice this problems could be reduced and overall user and administrator experience of Debian Systems could be improved.

    Policy Proposal

    System-wide Deliverymethod

    The file /etc/default/mailbox indicates the default delivery method for Email messages to the users mailbox. It uses a KEY=VALUE format.

    The actually defined values for the "INBOX_FORMAT" key and their meaning are:

    mbox
    delivery to a traditional unix /var/mail/<user> spool.
    Maildir
    delivery to a standard Maildir ~<user>/Maildir in the users home directory
    MH
    delivery to a Mail Handling mail directory in the users home directory.
    mbx
    indexed mbox used by wu-imapd and several MUA's.
    Other values can be used. If an official Debian package uses a different value this policy has to be updated to reflect the meaning of it.

    All packages which deliver Email messages to a users mailbox use this setting in their postinstallation script, to set up their default delivery methods, if apropiate.

    All packages which read or process messages from a user's mailbox use this setthing in their postinstallation script, to set up the default location from where to read the users messages.

    If the file does not exist, has unintelligable contents or the INBOX_FORMAT value is not known by a particular aplication, standard mbox delivery to /var/mail/user can be supposed.

    Package "defaultdelivery"

    A package "defaultdelivery" has to be created, which installs the /etc/default/mailbox file.

    "defaultdelivery" is an otherwise empty package, with just the task to define the default local delivery method.

    Thus it provides a postinstall script, asking the user with a priority of medium which defaultdelivery method she prefers for the Email system setup.

    All packages which either deliver mail to the user mailbox or process the user mailbox on a system-wide basis "depend" on the package "defaultdelivery".

    A script "update-defaultdelivery" is provided, which checks for installed packages dependent on "defaultdelivery" and calls their postinstall scripts.

    Mailhandling package requirements

    Packages which deliver Mail to user mailboxes, or which retrieve or process mail from them are called mailhandling packages. They have to comply to the following rules:

    1. The postinstallation script tests for the presence or absence of the file /etc/default/mailbox. If it does not exist, an error messages is issued and /var/mail/ delivery is supposed.
    2. If /etc/default/mailbox exists, the value of the INBOX_FORMAT key is used to determine the default delivery method.
    3. If the package cannot handle the default delivery method a warning message is issued and no default configuration is done.
    4. If the package is able to process Email messages acording to the default delivery method and can be used on a system-wide basis, the postinstallation script has to setup the corresponding configuration. If an entry in /etc/skel is required for the default configuration it should be made, accompanied by a warning message, indicating that the existing user accounts will be unaffected, and how to reconfigure the existing user accounts.
    5. If a package is not able to provide a system wide configuration, a warning message has to be issued, indicating how the individual user configuration has to be acomplished.

    Virtual mailhandling packages (optional)

    The following virtual packages are defined:

    mail-transport-agent
    receives Email messages from other systems or locally, and arranges for transport to its recipients, local o remote.
    mail-delivery-agent
    eventually preprocesses Email messages and delivers them to a local user mailbox.
    mail-reader
    retrieves Email messages upon command of a particular user for visualization and/or processing.
    mail-user-agent
    provides mail-reader and is capable of injecting messages, prepared by the user into the mail system.
    mail-notification-agent
    scans a particular users mailbox for the presence and status of Email messages and signals them to the user.
    mail-transport-agents may provide also mail-delivery-agent. If this is not the case, the package has to depend on a mail-delivery-agent. Mail delivery agents need not conflict with each other.

    Transition

    [Version a - provided * is included]

    The transition period is determined with the release of Sarge. During this period the following rules apply:

    All packages in testing providing any of the virtual mailhandling packages have to comply to this policy.

    Packages in stable do not need to comply with this policy.

    Non-conforming mailhandling packages can be used in any distribution branch, but cannot provide any of the virtual mailhandling packages.

    After the transition period, mailhandling packages in stable have to comply with this policy.

    [Version b, without the optional chapter *]

    All packages providing mail-reader or mail-transport-agent in testing have to comply to this policy.

    Respective packages in stable do not need to comply with this policy.

    Non-conforming mailhandling packages can be used in any distribution branch, but cannot provide either mail-reader or mail-transport-agent.

    After the transition period, mailhandling packages in stable have to comply with this policy.

    Steps to implementation

    1. Creation of the "defaultdelivery" package.
    2. Adoption of the policy for at least exim, login and mailx
    3. Support for the policy in debhelper and lintian.

    Different rationale

    The proposal gears towards an automatic "plug and play" configuration of the messaging system. Actually there are two hinderances:

    1. Reconfiguration from the standard /var/mail/ spool to any other system is only managable by hand.
    2. Various mail user agents do not support a system wide configuration.
    By requiring at least a message at installation time, the second issue is brought to awareness.

    The first issue can be resolved, by providing postinstallation scripts which use the contents of the /etc/default/mailbox file. By issuing dpkg-reconfigure of mailhandling packages they can adapt their configuration to a new delivery method, in case the system adminstrador changes her mind. The entry of "defaultddelivery" in the Depends: field of mailhandling packages serves as a registry for an automatic reconfiguration.

    The term "mail-reader" can be applied as well to a standard mail-user-agent with composing facilities, as to a mail-formatter like muttprint. So the well know term mail-user-agent is introduced to mark the difference between simple readers, and reader/composer programs. On a workstation setup it should suffice to install on implementation of each of the three virtual packages: mail-transport-agent, mail-delivery-agent, mail-user-agent, to be able to read, write and deliver locally and remotely Email-messages, which is the expected behaviour.

    On the other hand, systems are conceivable, which do not use Email, so the defaultdelivery package, as well as the virtual mailhandling packages should not be marked es required or even essential. Think of routing/firewalling computers or embeded systems.

    A user may decide to use a filter program to deliver her messages into different mailboxes, which need not necessary use the same format as the default delivery method of the system. Therefore it is not erroneous to install a utility which only has support for a mailbox format that is not supported by the default delivery method, and the policy implicitely allows it.

    Miquel van Smoorenburg proposes the following extensions to /etc/default/mailbox:

    INBOX_PATH
    Where to deliver messages, example: INBOX_PATH=~/Maildir/
    FOLDER_FORMAT
    In which format are the (other "standard"?) mail folders, example: FOLDER_FORMAT=Maildir
    FOLDER_PATH
    Where (in the filesystem) are the mail folders, example: FOLER_PATH=~/Mail
    This proposal can overcome the troubles with MUA configuration mentioned in *.

    Tecnical details

    Mail user agents pittfalls

    Traditional console mode mail user agents derive the location of the default system mailbox of the user from the environment variables MAIL. This is set traditionally by the login program of the shadow suite, or by the display manager in case of the X-Window grafical environment.

    The proposal tries to remedy the situation, that these variables have to be reconfigured by hand with each program that realises user session setup.

    However, the situation with grafical mail user agents is worse. These generally require an individual configuration by hand for each user. I guess that this situation comes from the user friendly grafical operating system provided by our friend Bill Gates, which has no concept about different users and Email as a standard system service. Windows MUA's use exclusively remote mail retrieval via IMAP, POP3 or similar non-standard mecanisms, and transport via SMTP (or similar...) on behalf of a complete lack of a system mailbox per user.

    As the amount of installed X-Windows Workstation is constantly growing, which have to use a mail-transport-agent by definition to provide a comunication mean between various system services and the (administrating) user, this approach seems unnecessarily complex: Why use a network connection to read a file on the local harddisk?

    One problem with the default configuration

    of grafical user agents is, that their startup configuration sometimes does not support environment variables, so the use of $MAIL to communicate the location of the users mailspool is not aplicable.

    Other problem is, that tilde expansion: "~user" is substituted by a string indicating the users home directory, is also not understood by many of these programs. While traditional unix mailspool paths can easily be built with variable substitution by using the loginname of the user /var/mail/<user>, Qmail and MH style Maildirs are formed by <homedir>/<maildir>.

    Finally, the traditional "/etc/<aplication>" and ".rc" file approach for system wide and user runtime configuration seems not be supported by some of these programs, at least the Evolution manpage and help text gives no information about it, and some previous version of Balsa I chose to evaluate was discarded because of similar problems.

    For the sake of a wider acceptance, easier deployment and improved efficiency a call shall be made here to all MUA designers, packagers and maintainers to overcome the respective flaws in the program they are responsible for. Hopefully the policy proposal also helps to "sort" them out, and thus challenge users to provide respective patches.

    Another issue are the different mailbox nameing conventions

    used by different mailers. The standard unix mailer uses the system mailbox as inbox and saves read messages into a file in the users home directory called mbox. Optionally a user can create mailboxes by saving messages to files with different names. Nowadays mail programs have ample support for these so called "mailfolders", but present the problem, that they are stored anywhere the application pleases, makeing it impossible to read ones mail with different mail user agents without heavy tweaking.

    "Modern" mailers use a different, sometimes very irritating approach, with respect to the system mailbox, which stems from the same mono-user environment mentioned above: The users system mailbox is sucked out completely and all messages are put into a folder, colled the "Input tray", or "Inbox" or the like.

    This behaviour makes new messages disappear for standard mailers, like pine, mutt, etc. which could be resolved by pointing these mailers to the "Input tray" of the "modern" mail program. Let's enumerate my experience:

     

    Kmail:
    ~/Mail/inbox, .../outbox, .../sent-mail, .../drafts
    Mutt:
    ~/Mail/...
    Balsa:
    $HOME - but does not expand ~, /home/<user>/mail/outbox, .../sentbox, .../draftbox, .../trash
    Evolution:
    ~/evolution/local/Inbox/mbox, .../Outbox/mbox, .../Sent/mbox, .../Drafts/mbox
    Attribute handling is done also in a very different ways, so that the simultaneous use of different MUA's on a Mailbox can cause corruption of the contents.

    Also this problem should be tackled at least in a way that make compatible programs work on the same folders and incompatible ones respect the contents of the others. It is not perceivable, that a user trying out other MUA need to reconfigure it to get some results, and eventually get her actual mail scrambled.

    List of (potential) mailhandling packages

    The following lists were elaborated by executing "apt-cache search mail" and than hand-sorting the result, without going into much depth. So there may be packages left out, and there may be packages included which do not belong here.

    The inclusion criteria is, if a package at any time of its normal operation touches a user mailspool, be it by delivering, or be reading Email messages from the spool. Another criteria is, if the package is involved in the Mail delivery setup of the system, as is the case with "login".

    MTA's:
    courier postfix sendmail nullmailer cyrus zmailer masqmail ssmtp exim smail qmail
    MDA's/Input Filters:
    filter spamassassin sanitizer scanmail mailfilter mailscanner tmda autorespond courier-maildrop junkfilter pycmail spamfilter mimefilter emil deliver maildrop sortmail procmail mailagent
    MUA's:
    sylpheed gnumail mailutils mutt pyne althea vm libroxen-webmail wl pronto cronosii gmail gnus postilion xmh af aethera mew nail tkmail exmh nmh balsa mailx kmail tkrat evolution mozilla-mailnews
    Output Filters:
    kuvert
    Notification tools:
    gbiff melon xlassie ixbiff gkrellm-mailwatch sonicmail glbiff youbin wmbiff gbuffy xmailbox xlbiff xfaces asmail mailleds xbuffy coolmail wmmail biff korn
    Remote retrieval:
    getmail gotmail fetchmail
    pop/imap-servers:
    teapop uw-imapd cyrus solid-pop3d courier perdition qmail
    Miscelanea:
    grepmail archivemail libmail-audit-perl libmail-box-perl libmail-pop3client-perl userinfo barrendero premail pms xfingerd splitdigest mboxgrep maildir-bulletin bbmail im grepmail vacation ixync mailsync checkpw
    This makes about 110 packages, which have to be modified to comply to the policy proposal.

    It would be nice to create tables, where the capabilities of each packages with respect to the different mailbox formats are shown, something like:
    Package mbox Maildir MH
    mutt * * *
    balsa *
    nmh *
    mail *

    Notes

    I am not sufficient intimate with the naming conventions to get all the terms right. The intended use of this instruction is, that in the pre-configuration stage the respective options can be set by the installing user. If the term "postinstallation script" is erroneous in this respect I ask humbly for a correction of my wording.

    UUCP, Netnews, BBS Mail and Packet Radio Mail systems and programs have been let out knowingly. In first place because I know virtually nothing about them. Please step in if you have to contribute something with respect to these messaging systems, it would of course be nice to be able to integrate them into the policy if aplicable.

    This proposal is one of various, which are intented to improve the messaging environment and elevate them to a state of the art quality product. One of my goals is to introduce the /etc/mta switch proposed by D. J. Bernstein or any similar approach into Debian. The other is to provide means to allow different mail transport agents to be installed in a system at the same time. This allows for alternative Messaging systems to be introduced in parallel or as a substitute to the existing one(s), see the im2000 or Qim initiative as an example.

    Credits

    Peter Palfrader helped me correct some misspelings, ~ notation errors and reference the X Window System correctly .

    Miquel van Smoorenburg proposed the better name /etc/default/mailbox and to keep with the KEY=VALUE format used with other files in /etc/default/. He also pointed out the existence of the mbx mailbox format.

    Newton, Isaac, told me that standing on the shoulder of giants one can see very far. The Internet comunity made this wisdom available to me and the Email-giants gave me all kind of electronic message systems to chose from them. Special thanks to the authors of Qmail, Exim and Sendmail.


    Georg Lehner - homepage

    IntroductionTopContents