Implementation

Implementation

Exploiting Qmail

Although I am not of the opinion, that there should be much effort in making gateways to traditional Email this implementation makes heavy use of it.

Qmail has very powerful features, which lends it to the implementation of:

1. a preliminary notification mecanism for im2000
2. a posting and store mecanism for im2000 messages, described here.

The second feature is the dot-qmail delivery mecanism. Messages to im2000 are delivered acording to instructions an the file .qmail in the home directorio of im2000. Messages to im2000-token are delivered according to .qmail-token. This mecanism is recursive: .qmail-token1-token2 handles im2000-token1-token2. There is a catch-all mecanism: .qmail-token1-default handles im2000-token1-anything_else, and .qmail-default handles im2000-anything_else. Inside of .qmail files, messages can be forwarded, stored and passed over to programs.

While this implementation uses a specific im2000 account, it should be trivial to use:

1. a per user -im2000- extension mecanism. Any user account user can install a user-im2000 message store server and even multiple user-virtual-im2000 servers.
2. Host an arbitrary amount of im2000 message store servers on an ISP by creating one account for each server (which is actually the same as 1.)

Message Store

This implementation uses the home directory of an account named im2000 to store "outgoing" im2000 messages. When a client wants to create a Message Thread it sends a traditional Email to im2000­nt. The "server" creates a uniquely named Maildir and sends a tradition Email reply to the client, containing the Message Thread ID composed of the name of the Maildir and other information.

To add a message, the client sends a traditional Email to im2000-add-sequence. This message contains in the Subject: line the Message Thread ID. The body is the message, eventually mime encoded. Only the first part of the body is taken into account in a multi-part mime message. The recipient identifiers are coded into aditional header lines denoted with X-im2000-recipient-identifier: There is no asumption made about the contents of this field. The server sends an Email reply to the client, with the complete message ID in the Subject: line.

To indicate end of a message block, the client sends a Email to im2000-eb, with the Message Thread ID in the subject.

Client autentication

Anybody from anywhere would be allowed to post messages, to im2000 on a open posting mailing list server. Of course, traditional RBL mecanisms, antispam-filtering etc. would of course be apliable.

Other aproach would be to disable complete remote incoming mail (qmail-smtp). Only local clients could post messages. This corresponds to a normal workstation operation. The collection mecanism could not rely on a similar implementation then.

TMDA - Tagged Message Delivery Agent seems an ideal solution for a "Right Now" public im2000 Email service provider setup. With Qmail, TMDA uses the same dot-qmail mecanism as posting does and they could be combined easily. Each local client of the ISP would be whitelisted inside the im2000 accounts tmda setup - remote colectors would have to autenticate via dated or user Sender Address tags, anything else would be blacklisted. Also notification and message collection could (easily?) be implemented via a similar mecanism, using TMDA for remote user and notify/collect message identification and autentication.

New thread

im2000-nt@host

or

.qmail-nt

 

im2000-virtual-nt

or

.qmail-virtual-nt

implements the

new_thread

function

1. A client wanting to create a Message Thread sends a traditional Email to im2000­nt.
2. [optional] The server checks, if the sender of the email is allowed to create a thread, and if there is room to do it. If any of this conditions fails, an empty reply is sent to the client.
3. The `"server`" spools the message in a directory named threads using safecat, tmp is used as a temporary directory for the spoolin. This way, automátically a unique name for the Message Thread is created by the name of the file where the im2000-nt message is spooled. safecat tells the name of the Thread on stdout.
4. [optional] The Message Thread ID is enriched with cryptographic hashes, challenges, etc, involving the im2000-nt message, it's senders identity, secrets, etc. Also a message body may be generated. It may be necesary to include credentials of the server, to autentify itself to the client.
5. The "server" composes a Reply Mail with the Message Thread ID in the subject line.

| THREAD=$(/usr/local/bin/safecat tmp threads); \

  /usr/bin/maildirmake $THREAD ; \

  echo $MESSAGE | mail -s `"$THREAD`" $SENDER

#

# 1) tmda processing here

# not done

# 2) and here, is there room for messages?

# not done

# 3) create the threadID

# | THREAD=$(/usr/local/bin/safecat tmp threads); \

# /usr/bin/maildirmake $THREAD ; \

# 4) autenticate and secure the reply

# not done

# 5)

# echo $MESSAGE | mail -s `"$THREAD`" $SENDER

• In 4) no check of safecat and maildirmake exit status is made at all. man safecat mentions means to use it in a secure way in shell scripts - any volunteers to rewrite this piece of code accordingly?
• Is $SENDER or $RPLINE - the Return-Path - the right thing to reply to?
• CLIENTEXT was ment to be used for construting the Reply address somehow like: NAME-EXT@HOST to allow the qmail-extension mecanism work for us on the client side.
• in 5) mail should be replaced by qmail-inject, I think. Or better, something like qreceipt should be rewritten. It would be nice, if we reused the MessageID field of the original mail, so that the client has an easier job to keep track.
• Efficiency considerations have to be done, especially with respect to the home directory filling up with Maildir's. Normal unix filesystems slow down significantly directory lookup with lots of directory entries. Eventually a Btree Hierarchy should be created, but then, on the other hand this is only an experimental testbed implementation which should not be considered a model for real implementations - so with bother too much.

Add Message

im2000-am-sequence

or

.qmail-am-default

and a symbolic linkt to it

.qmail-am

implement the

add_message

function

1. The client sends a traditional Email to im2000-add-sequence. This message contains in the Subject: line the Message Thread ID. The body is the message, eventually mime encoded.
2. [optional] The server checks, if the sender of the email is allowed to add messages, to add them to this thread, and if there is room to do it. If any of this conditions fails, an empty reply is sent to the client.
3. The server extracts the Message Thread ID and delivers the Email to the Maildir with the same name via savecat, so the Message ID can be retrieved. (free space?)
4. The server moves the message to Maildir/cur, appending ":3,sequence" to the name. Note that this as a deliberate extension of the Maildir protocol. :3 is not specified there, I will have to ask djb for permissions to do that ;-)
5. [not implemented ] The server notifies the client with a failure message, if a duplicate secuence number is requested, further processing is abandoned, the message is considered not posted and should be deleted by the server.
6. [optional] The Message ID is enriched with cryptographic hashes, challenges, etc, involving the im2000-add-secuence message, it's senders identity, secrets, etc. Also a message body may be generated. It may be necesary to include credentials of the server, to autentify itself to the client.
7. The server composes a Reply Mail with the Message ID in the subject line.

| MESSAGEID=$(/usr/local/bin/safecat tmp mail) \

  ; MAILDIR=$(/usr/local/bin/822field < mail/$MESSAGEID) \

  ; MESSAGEID=$(/usr/local/bin/safecat $MAILDIR/tmp $MAILDIR/cur \

                  < mail/$MESSAGEID) \

  ; mv $MAILDIR/cur/$MESSAGEID $MAILDIR/new/$MESSAGEID:3,$EXT2 \

  ; echo | mail -s `"$MESSAGEID:3,$EXT2`" $SENDER

#

# 1) tmda processing

# not done

# 2) Check if there is space

# not done

# 3a) save the message

#     | MESSAGEID=$(/usr/local/bin/safecat tmp mail) \

# 3b) extract message thread id

#     ; MAILDIR=$(/usr/local/bin/822field) < mail/$MESSAGEID) \

# 3c) and spool the message

#     ; MESSAGEID=$(/usr/local/bin/safecat $MAILDIR/tmp $MAILDIR/cur \

                  < mail/$MESSAGEID) \

# 4) store the message with it's corresponding sequence number

#     ; mv $MAILDIR/cur/$MESSAGEID $MAILDIR/new/$MESSAGEID:3,$EXT2 \

# 6) add cryptography and autentication

# not done

# 7) acknoledge storage of the message

#     ; echo | mail -s `"$MESSAGEID:3,$EXT2`" $SENDER

• I would rather prefer to reply to client-secuence@host, but have not taken the challenge to find out how to rewrite the address easily.
• Return-Path doubts from im2000-nt apply here too.
• also the the CLIENTEXT from im2000-nt applies.

End Block

im2000-eb

or

.qmail-eb

implement the

end_block

function

It's out of scope of this implementation to decide if the server should bundle notifications of multiple messages, message threads and even message threads from diferent posters to a Recipient Identifier. The server may even pre-notify about any message (parts) yet posted, without waiting for the End Block signalisation.

For demonstration and testing purposes we construct a notify message to the poster, which shows all messages of the thread.

| THREAD=$(/usr/local/bin/822field) ; \

  for i in $THREAD/cur/*; do \

    /usr/local/bin/822header < `"$i`" ; \

  done \

  | mail -s `"im2000 thread: $THREAD`" $SENDER

Notes

• Could Gateways to traditional Email render Spam reduction mecanisms of im2000 useless, or take advantge of it by using a more reliable delivery mecanism?
• It seems natural, to do notification with "maildirserial thread-id " notify" where "notify" are different maildirserial clients. An instant notifying client which runs with im2000-eb, a periodic notifier which runs with cron and redoes notifying when a message has not been colected, a expirer which checks expiring dates, etc.

Implementation