Delete duplicate e-mail messages

If you need to delete duplicate e-mail messages on an IMAP server, take a look at this useful IMAP de-duplicator script:

IMAP de-duplicator – IMAPdedup

As IMAPdedup is a command line interface tool (a python script), it’s particularly useful for:

  • automated deletion of duplicates (as it can be called from other scripts)
  • extraordinarily big mailboxes or if you have many subfolders (as there’s no intervention by the user required)
  • if you have console/shell access to the IMAP server (as you can then run the script on the server itself, speeding the de-duplication process further up)

I also found that it deals relatively well with failures (e.g. when a folder is read-only and hence messages can’t be deleted): It simply reports them on the screen and carries on.

Here’s a quick’n’dirty bash script to de-dup the inbox and all subfolders of the specified account:

#!/bin/sh
# Delete all duplicate messages in all folders of said account.
# Note that we connect through SSL (-x) to the default port.

SERVER="my.server.com"
USER="mylogin"
PASS="mypass"

for folder in `imapdedup.py -s $SERVER -x -u $USER -w $PASS -l`;
do
 imapdedup.py -s $SERVER -x -u $USER -w $PASS $folder
done

If you only have to de-duplicate messages in a small folder, you could also use the following de-duplication add-on for Mozilla Thunderbird:

Remove Duplicate Messages Add-on for Thunderbird

Note however that the ‘Remove Duplicate Messages’ add-on is intended for interactive use only, not for batch processing. I also noticed that it fails at cleaning big mail folders (e.g. containing 50’000 messages).

 

Thunderbird Add-on: S/MIME Security for Multiple Identities

I’ve just found and installed the following add-on for Mozilla Thunderbird:

S/MIME Security for Multiple Identities

It allows you to use a different S/MIME certificate for each of the different identities (i.e. “e-mail address aliases” or “profile aliases”) you defined in your Thunderbird profile.

It’s currently still marked as an experimental add-on and I’ve noticed a minor glitch in v0.3.0 when using it (see my add-on review), but this might also be related to the fact that I also use the Virtual Identity add-on (another nice add-on which allows you to use an arbitrary sender address for sending messages).

The “S/MIME Security for Multiple Identities” add-on is very convenient if you have multiple e-mail accounts and want to use S/MIME message signing and/or encryption with all of them.

Bonus hint: You can get your own, personal S/MIME certificates for free at Thawte (for e-Mail only) or StartCom/StartSSL (also offers free class 1 SSL/TLS certificates for FTP servers, web servers etc. -> the latter don’t “work” with Internet Explorer, however).