In the current setup of the basic mailserver all communication between your server and the end user email client is unencrypted and there is the possibility that usernames and passwords can be sniffed while in transit on the internet. This can be prevented by using TLS/SSL encrypted connections which will encrypt the traffic between the client and the server which in turn means that snooping of password information is history.

First you need to buy yourself a SSL certificate at Thawte or Verisign, but as we are building a server on the cheap we are going to create our own certificate. The only problem you will encounter when creating your own certificates is that users explicitly have to accept and verify your root certificate in contrast with certificates you buy which are already accepted in most email clients by default. If they for instance try to send their email for the first time via your secure server they need to accept your certificate. When using Mail.app in OS X they will get the following warning:

Unable to verify certificate

They need to press continue and from then on your certificate will be accepted and they won’t be asked again.

Just open a Terminal and execute the following command in the directory ‘/etc/postfix‘:

sudo openssl req -new -outform PEM -out smtpd.cert \
    -newkey rsa:2048 -nodes -keyout smtpd.key -keyform PEM \
    -days 365 -x509

This will create a 2048 bit encryption key that, for now, is secure enough for you mailserver to use. If you are paranoid and want a bigger key just increase the number after rsa:. The key will be valid for a year, if you want a longer period just increase the number after the -days option. When the key is finished you will be asked a couple of questions you need to answer. The information will be shown to people who want to see your certificate when their mail client complains. The most important one is the ‘Common Name’, make sure that that one is the same as the mail server name.

Country Name (2 letter code) [AU]:
State or Province Name (full name) [Some-State]:
Locality Name (eg, city) []:
Organization Name (eg, company) [Internet Widgits Pty Ltd]:
Organizational Unit Name (eg, section) []:
Common Name (eg, YOUR name) []:your.mailserver.tld
Email Address []:you@yourdomain.tld

 

Configuring Postfix

Now you can configure Postfix to make use of it and to enforce the usage of TLS to securely communicate with the email client. You’ll have to add the following lines to the configuration file ‘main.cf‘ in ‘/etc/postfix‘:

smtpd_enforce_tls = no
smtpd_tls_loglevel = 1
smtpd_use_tls = yes
smtpd_tls_key_file = /etc/postfix/smtpd.key
smtpd_tls_cert_file = /etc/postfix/smtpd.cert

Issue the command ‘sudo postfix reload‘ to refresh the configuration of your mail server and your ready to test it out. Start a terminal session and issue the following commands:

telnet your.mailserver.tld 25

The server will answer with:

Trying your.mailserver.tld…
Connected to your.mailserver.tld.
Escape character is ^]
220 your.mailserver.tld ESMTP Postfix

Then type in:

EHLO your.mailserver.tld

And again your server will answer it’s capabilities:

250-your.mailserver.tld
250-PIPELINING
250-SIZE 10240000
250-ETRN
250-STARTTLS
250 8BITMIME

Now it’s time to test TLS and enter in capitals:

STARTTLS

and the server should respond with:

220 Ready to start TLS

Then you know it will work, you could give your favourite email client a try.
 

Configuring Dovecot

The proces for securing the IMAP server is the same as for postfix, first generate a certificate and configure Dovecot to use that certificate with the appropriate settings.

Dovecot comes with a script to build self-signed SSL certificates using OpenSSL. The SSL certificate’s configuration is taken from the file ‘dovecot-openssl.cnf‘ file in the doc directory of the sourcecode. You need to edit that file and change the following settings:

default_bits = 2048
 
[ req_dn ]
# country (2 letter code) (your country)
C=NL
 
# State or Province Name (full name, not required)
#ST=
 
# Locality Name (eg. city)
L=Rotterdam
 
# Organization (eg. company)
O=DIYMacServer
 
# Organizational Unit Name (eg. section)
OU=IMAP server
 
# Common Name (*.example.com is also possible)
CN=*.richard5.tld

As with Postfix the important field is the CN (Common Name) field, which should contain your server’s host name. The clients will verify that the CN matches the connected host name, otherwise they’ll say the certificate is invalid. It’s also possible to use wildcards (eg. *.domain.com) in the host name. They should work with most clients.

If you are done than you need to edit the script that creates the SSL certificates called ‘mkcert.sh‘. This is needed as there are some setting we like differently.

Make the following changes to the script:

SSLDIR=${SSLDIR-/etc/dovecot}
 
CERTDIR=$SSLDIR
KEYDIR=$SSLDIR
 
CERTFILE=$CERTDIR/dovecot.pem
KEYFILE=$KEYDIR/dovecot.pem

When you are finished you can execute the script:

chmod a+x mkcert.sh
sudo ./mkcert.sh

If everything went correctly you should see a file ‘dovecot.pem‘ in the /etc/dovecot directory.

Now we need to configure Dovecot to use the newly created certificates. You need to edit the file ‘dovecot.conf‘ in the /etc/dovecot directory and change the following settings:

# Protocols we want to be serving: imap imaps pop3 pop3s
# If you only want to use dovecot-auth, you can set this to “none”.
protocols = imap imaps pop3 pop3s
 
# Disable LOGIN command and all other plaintext authentications unless
# SSL/TLS is used (LOGINDISABLED capability).
disable_plaintext_auth = yes
 
# SSL/TLS support: yes, no, required.
ssl = yes
 
ssl_cert_file = /etc/dovecot/dovecot.pem
ssl_key_file = /etc/dovecot/dovecot.pem

To activate the new configuration you need to kill the running Dovecot process. If you used the previous discussed launchd plist instructions Dovecot will restart automatically.

No next step defined yet, you got a very good secure mailserver running now.