Search:  
Gentoo Wiki

Howto_Postfix_virtual_mail_server_with_IMAP_POP3_SpamAssassin_Virus_Scanner_and_Roundcube_webmail

Split-arrows.gifIt has been suggested that this article be split into multiple articles accessible from a disambiguation or index page.   (Discuss)

This walkthrough assumes you have working versions of PHP, Apache and MySQL.

Contents

Goal

This tutorial aims to have a fully functional MTA with virtual mail, spam filtering, and webmail. With Roundcube as our webmail we can easily move messages in and out of a Junk folder that maildrop and SpamAssassin will sync with. A nightly cron will retrain SpamAssassin based off of what is in the Junk folder and what is not.

Using courier-authlib we can check against the system's shadow file as well as a mysql database for user authentication. Local users will enjoy the convenience of not having to supply their full email addresses for imap/pop3/webmail login.

Note: POP3 clients like to erase all inbox messages from the server. There is no problem with this, but it will raise the learning curve for SpamAssassin since there won't be any legitimate email for contrast.

This walkthrough is constructed in a way that related tasks are lumped together.

What we're merging and why

Our MTAs, Antivirus, and Spamfilter:

emerge -av postfix courier-imap amavisd-new spamassassin

Pull down maildrop for mail delivery as well as server-wide message rules. This will help sync SpamAssassin and Roundcube

emerge -av maildrop

Configuring php to play nice with Roundcube

echo "dev-lang/php cli imap apache2 mysql" >> /etc/portage/package.use
emerge -av dev-lang/php PEAR-PEAR

At this point it would be a good idea to reload Apache and make sure your PHP hasn't broken.

/etc/init.d/apache2 reload

Add the vmail user

The vmail user will be our go-between for keeping track of virtual users and their mail.

useradd -m -s /bin/false -d /home/vmail vmail
passwd vmail

vmail's UID and mail's GID will play a role later on, so get those and write them down.

id vmail

Mail for virtual users will be stored in /home/vmail/virtualdomain.tld/user/mail

To create a maildir for user@virtualdomain.tld

mkdir -p /home/vmail/virtualdomain.tld/user
maildirmake /home/vmail/virtualdomain.tld/user/mail
chown -R vmail:mail /home/vmail/virtualdomain.tld
chmod -R o-rwx /home/vmail/virtualdomain.tld

Setting up databases

mysql -uroot -p

Aliases and Virtual Email database

create database vmail;
use vmail;
grant all on *.* to vmail identified by 'password';

create table aliases (
 alias varchar(255) not null primary key,
 crypt varchar(255) not null,
 destination varchar(255) not null,
 modified timestamp
);

Roundcube's Workspace

create database roundcubemail;
use roundcubemail;
grant all on *.* to roundcube identified by 'password';

SpamAssassin's Workspace

create database spamassassin;
use spamassassin;
grant all on *.* to spamassassin identified by 'password';

-- bayes database structure
-- taken from http://spamassassin.apache.org/full/3.0.x/dist/sql/bayes_mysql.sql

CREATE TABLE bayes_expire (
 id int(11) NOT NULL default '0',
 runtime int(11) NOT NULL default '0',
 KEY bayes_expire_idx1 (id)
) TYPE=MyISAM;

CREATE TABLE bayes_global_vars (
 variable varchar(30) NOT NULL default ,
 value varchar(200) NOT NULL default ,
 PRIMARY KEY  (variable)
) TYPE=MyISAM;

INSERT INTO bayes_global_vars VALUES ('VERSION','3');

CREATE TABLE bayes_seen (
 id int(11) NOT NULL default '0',
 msgid varchar(200) binary NOT NULL default ,
 flag char(1) NOT NULL default ,
 PRIMARY KEY  (id,msgid)
) TYPE=MyISAM;

CREATE TABLE bayes_token (
 id int(11) NOT NULL default '0',
 token char(5) NOT NULL default ,
 spam_count int(11) NOT NULL default '0',
 ham_count int(11) NOT NULL default '0',
 atime int(11) NOT NULL default '0',
 PRIMARY KEY  (id, token)
) TYPE=MyISAM;

CREATE TABLE bayes_vars (
 id int(11) NOT NULL AUTO_INCREMENT,
 username varchar(200) NOT NULL default ,
 spam_count int(11) NOT NULL default '0',
 ham_count int(11) NOT NULL default '0',
 token_count int(11) NOT NULL default '0',
 last_expire int(11) NOT NULL default '0',
 last_atime_delta int(11) NOT NULL default '0',
 last_expire_reduce int(11) NOT NULL default '0',
 oldest_token_age int(11) NOT NULL default '2147483647',
 newest_token_age int(11) NOT NULL default '0',
 PRIMARY KEY  (id),
 UNIQUE bayes_vars_idx1 (username)
) TYPE=MyISAM;

Verify that all logins work before continuing.

Notes on the vmail database

In the vmail database a forward is defined by a blank crypt field. You can forward anything any which way. Anything without an @domain on it will be interpreted as a local user. The following configuration is a daisy chain of forwards, starting with a catchall, all leading straight to joe's localhost mailbox. Note: Forwarding to a forward is bad practice! This is merely here to demonstrate a complex configuration!

mysql> select alias,destination from aliases where crypt="";
+---------------------------+---------------------------+
| alias                     | destination               |
+---------------------------+---------------------------+
| @virtualdomain.tld        | virtual@virtualdomain.tld |
| virtual@virtualdomain.tld | virtual@anotherdomain.tld |
| virtual@anotherdomain.tld | joe                       |
+---------------------------+---------------------------+

Here are more upstanding examples, keeping forwarding single-leveled.

+-----------------------+------------------+
| alias                 | destination      |
+-----------------------+------------------+
| dave                  | dave@virtual.tld |
| bob@virtual.tld       | bob@another.tld  |
| webmaster@virtual.tld | john             |
+-----------------------+------------------+

Virtual mailbox entries in the vmail database will look something like this.

mysql> select alias,crypt,destination from aliases where crypt!="";
+------------------+------------------------------------+-------------------------------+
| alias            | crypt                              | destination                   |
+------------------+------------------------------------+-------------------------------+
| dave@virtual.tld | $1$1ZAEvOvu$oIrrkJUAE4TPdLK93St571 | /home/vmail/virtual.tld/dave/ |
| bob@another.tld  | $1$pYB1SOWo$bekEPusqEN0s.llW9PMc5/ | /home/vmail/another.tld/bob/  |
+---------------------------------------------------------------------------------------+

Configuring Postfix

Remember, deep breaths.

/etc/postfix/main.cf

# myhostname is the primary FQDN of your mailserver
myhostname = example.com
mydomain = $myhostname
mydestination = $myhostname, localhost
mynetworks_style = host

# allow mail for domains that are not $myhostname
virtual_alias_domains = virtualdomain.tld, anothervirtualdomain.tld, etc
virtual_mailbox_domains = $virtual_alias_domains
#alias_maps = ...

# set up delivery
virtual_alias_maps = mysql:/etc/postfix/mysql-aliases.cf
virtual_mailbox_maps = mysql:/etc/postfix/mysql-mailboxes.cf
local_recipient_maps = 
mailbox_command = /usr/bin/maildrop -d "$USER" -f "$SENDER" "$EXTENSION"
fallback_transport = maildrop
virtual_transport = maildrop
home_mailbox = mail/

Add content-filter details at the bottom

content_filter = smtp-amavis:[127.0.0.1]:10025
biff = no
message_size_limit = 25000000
smtpd_helo_required = yes

# uncomment these (and remove the linebreaks) to be more strict
#smtpd_helo_restrictions = permit_mynetworks, reject_unauth_pipelining, reject_non_fqdn_hostname,
#                          reject_unknown_hostname, reject_invalid_hostname, permit
#smtpd_recipient_restrictions = permit_mynetworks, reject_unauth_destination,
#                               reject_multi_recipient_bounce, reject_non_fqdn_recipient,
#                               reject_unknown_recipient_domain

/etc/postfix/mysql-mailboxes.cf

# determine all real mail addresses
# this prevents catchalls from catching legitimate mail
user            = vmail
password        = password
dbname          = vmail
table           = aliases
select_field    = alias
where_field     = alias
hosts           = 127.0.0.1

/etc/postfix/mysql-aliases.cf

# determine forwards and catchalls
user            = vmail
password        = password
dbname          = vmail
table           = aliases
select_field    = destination
where_field     = alias
additional_conditions = and crypt=""
hosts           = 127.0.0.1

/etc/postfix/master.cf

maildrop  unix  -       n       n       -       -       pipe
  flags=DRhu user=vmail argv=/usr/bin/maildrop -d ${recipient}

Add amavis details at the bottom

smtp-amavis unix	-	-	y	-	4	smtp
 -o smtp_data_done_timeout=1200
 -o disable_dns_lookups=yes

127.0.0.1:10025	inet	n	-	y	-	-	smtpd
 -o content_filter=
 -o local_recipient_maps=
 -o relay_recipient_maps=
 -o smtpd_restriction_classes=
 -o smtpd_helo_restrictions=
 -o smtpd_sender_restrictions=
 -o smtpd_recipient_restrictions=permit_mynetworks,reject
 -o mynetworks=127.0.0.0/8
 -o strict_rfc821_envelopes=yes

Configuring SpamAssassin and Amavis

/etc/conf.d/spamd

SPAMD_OPTS="-m 5 -u vmail"
FIXME Is running spamd necessary since amavisd uses the SpamAssassin libraries directly? 
This directly contradicts the Gentoo Mailfiltering Guide (Configuring SpamAssassin section) in the Gentoo Official Documentation. 

/etc/spamassassin/local.cf

use_bayes 1
bayes_auto_learn 1
bayes_auto_learn_threshold_nonspam -1
bayes_auto_learn_threshold_spam 14.00
bayes_ignore_header X-Bogosity
bayes_ignore_header X-Spam-Flag
bayes_ignore_header X-Spam-Status

/etc/spamassassin/secrets.cf

#(Tell Spamassissin to use MySQL for bayes data
bayes_store_module              Mail::SpamAssassin::BayesStore::SQL
bayes_sql_dsn                   DBI:mysql:spamassassin:localhost:3306
bayes_sql_username              spamassassin
bayes_sql_password              password

/etc/amavisd.conf

$mydomain = 'example.com'; #fdqn of this server
$myhostame = 'example.com'; #fqdn of this server

Configuring maildrop

Maildrop is going to take any email marked as spam by spamassassin and place it into a Junk folder alongside Inbox etc. Roundcube will automatically pick up on this.

/etc/maildroprc

DEFAULT = "$HOME/mail"
MAILDIR = $DEFAULT

Add to the bottom of maildroprc

#Drop spam in junk folder
if      (/^X-Spam-Status: Yes/)
{
    exception {
        to $DEFAULT/.Junk/
    }
}

Configuring Authlib

Maildrop and IMAP/POP3 authenticate using whatever authlib specifies. Here we will specify to check our virtual users and fall back on local users.

/etc/courier/authlib/authdaemonrc

##NAME: authmodulelist:2
#Email boxes can belong to virtual and actual users.
authmodulelist="authmysql authshadow"

/etc/courier/authlib/authmysqlrc

Connection details

MYSQL_SERVER    localhost
MYSQL_USERNAME  vmail
MYSQL_PASSWORD  password

Database details (insert vmail's UID and mail's GID for these values, single quoted.

MYSQL_DATABASE      vmail
MYSQL_USER_TABLE    aliases
MYSQL_CRYPT_PWFIELD crypt
#DEFAULT_DOMAIN     ...
MYSQL_UID_FIELD     'vmail's UID'
MYSQL_GID_FIELD     'mail's GID'
MYSQL_LOGIN_FIELD   alias
MYSQL_HOME_FIELD    destination
#MYSQL_NAME_FIELD   ...

Configuring IMAP/POP3

/etc/courier-imap/imapd

Listen on all interfaces.

ADDRESS=0

Don't purge trash.

IMAP_EMPTYTRASH=

Mail is stored here:

MAILDIR=mail
MAILDIRPATH=mail

/etc/courier-imap/pop3d

Mail is stored here:

MAILDIR=mail
MAILDIRPATH=mail

Roundcube Webmail

Install Roundcube to /home/vmail/public_html

See Roundcube

Create webmail subdomains

Edit apache's vhosts file and add at the bottom:

# all mail
<VirtualHost *:80>
 ServerName webmail.somedomain.tld
 ServerAlias webmail.anotherdomain.tld
 ServerAlias webmail.andanotherdomain.tld
 #ServerAlias webmail.etc.etc
 ServerPath /usr/lib/apache2
 DocumentRoot /home/vmail/public_html
</VirtualHost>

Make sure the DNS for these domains have a 'webmail' CNAME set to your mail server.

Start your engines

rc-update add apache2 default
rc-update add courier-authlib default
rc-update add courier-imapd default
rc-update add amavisd-new default
rc-update add spamd default
rc-update add postfix default
rc-update add clamd default
/etc/init.d/apache2 reload
/etc/init.d/courier-authlib start
/etc/init.d/courier-imapd start
/etc/init.d/amavisd-new start
/etc/init.d/spamd start
/etc/init.d/postfix start
/etc/init.d/clamd start

If all services started without problems it is time to start testing. If you haven't already, it will make things easier for you if you use phpMyAdmin to administer the vmail database: phpMyAdmin uses MySQL to generate crypts, which are not the same as those found in shadow. A workaround is to generate one via command-line and then copy it.

php -r 'echo crypt("password")."\n";'

Open a couple terminals, one for tweaking settings if needed and one for monitoring the mail log. We will want to monitor the mail log's most recent entries:

less +F /var/log/mail.log

and

less +F /var/log/messages

Add some forwards and virtual mailboxes and send them some mail from external email addresses. From this point you can actively watch whats happening inside the server as it routes mail.

Visit your webmail locations to make sure they are all pointed correctly and that you can log in. Try sending mail between virtual and local users to make sure everything is working properly.

Routine maintenance

When adding new virtual domains you must update both /etc/postfix/main.cf and /etc/amavisd.conf. A nightly cron can be added to update clamav's virus definitions and train SpamAssassin.

~/spamlearn.sh

echo "Learning from local users:";
for user in $(ls -1 /home); do
 if [ -d /home/$user/mail/.Junk ]; then
  echo " - $user's spam"
  echo -n "   - "
  sa-learn --spam /home/$user/mail/.Junk/cur -u $user
  echo " - $user's ham"
  echo -n "   - "
  sa-learn --ham /home/$user/mail/cur -u $user
 fi
done
echo

echo "Learning from virtual users:";
for domain in $(ls -1 /home/vmail); do
 for user in $(ls -1 /home/vmail/$domain); do
  if [ -d /home/vmail/$domain/$user/mail/.Junk ]; then
   echo " - $user@$domain's spam"
   echo -n "   - "
   sa-learn --spam /home/vmail/$domain/$user/mail/.Junk/cur -u $user@$domain
   echo " - $user@$domain's ham"
   echo -n "   - "
   sa-learn --ham /home/vmail/$domain/$user/mail/cur -u $user@$domain
  fi
 done
done
echo
echo "Done."
chmod u+x ~/spamlearn.sh
crontab -e
# spamassassin learning every night at midnight
0 0 * * * ~/spamlearn.sh
# update clamav's virus definitions nightly
0 0 * * * freshclam
Retrieved from "http://www.gentoo-wiki.info/HOWTO_Postfix_Virtual_Mail_%2B_IMAP/POP3_%2B_Spamassassin_%2B_Roundcube_Webmail"

Last modified: Mon, 08 Sep 2008 20:21:00 +0000 Hits: 5,663