Search:  
Gentoo Wiki

HOWTO_Email_System_for_the_Home_Network_Part_2


This article is part of the HOWTO series.
Installation Kernel & Hardware Networks Portage Software System X Server Gaming Non-x86 Emulators Misc

Contents

Part I

Part 1 of this guide can be found here

Email clients

This guide will help you create a fully functional email service within a home network. You will run servers to allow you to both send and receive email from all over the world. We will use free services to facilitate this such as DynDNS.

This guide can be used as full blown mail server provided you have an MX record pointing to your mail server. No changes are necessary.

This guide was taken lock, stock, and barrel from here

KMail

Kmail is the default Email client that is emerged with KDE. It is a fully functional email client with support for every protocol we'll use and many we don't. In it's latest incarnation (3.2) it is lightning fast and very stable. It is the client I use and have the most experience with. For this setup, we'll assume you have already set up your Identities section.

  1. Create a new network Configuration (Settings->Configure Kmail) and make sure you're viewing the "Sending" tab.
  2. Click on "Add..." button. A new window will open offering you a choice of Sendmail or SMTP. Select SMTP.
  3. Choose a name to enter in the Name field. The host field should contain the [IP] of your server (in my example: 192.168.2.2).
  4. Check the box that says "Server requires authentication", then enter the username and password you have recorded in chart 2.1.
  5. Click on the "Security" tab and click the button that reads: "Check What the Server Supports." In my case it was TLS for encryption and PLAIN for authentication method. Now click OK.
  6. Still in the Networking Options, click on the Receiving tab. Now click "Add..." and when a new window opens up, choose "IMAP"
  7. Enter the exact same information you used before, same host, user and password.
  8. In the security tab, click on the button: "Check What the Server Supports" and wait until the options are changed. In my case it was "Use SSL for secure mail download" and "Plain" as the authentication method.

Click on the OK button and you're all set. Apply the changes and test the email by sending yourself an email and receiving it.

Sylpheed-Claws

Sylpheed-Claws is a GTK+ application based on the Sylpheed email client. It can be referred to as the extended version of Sylpheed. This is also the client preferred by several anti-bloat people over Evolution.

  1. First thing we do is create a new account (Configuration->Create new account...).
  2. Fill out the Personal Information and set a name for the account.
  3. In the server information, change Protocol to IMAP4.
  4. Change the server for both receiving and sending to [IP] or in my example: 192.168.2.2
  5. Fill in User ID and Password with the Server user's username and password found in chart 6 under "Local IMAP server"
  6. Under the "Send" tab, make sure to check off the SMTP Authentication (SMTP AUTH) option. Next fill in the the username and password found in chart 2.1 under the "Server" heading.
  7. Under the "SSL" tab, check off "Use SSL for IMAP4 connection" option and below that, check off Use STARTTLS command to start SSL session

Apply the changes and test the email setup by sending yourself an email and then check to see that it was received correctly.

Outlook Express 6

Outlook Express is the default email client used on most Windows Machine. It comes pre-installed on every Windows version and is freely updated at Windows Update. NOTE: that I cannot test what happens with the million and one types of anti-virus programs out there and their outbound email scanning. Since we've changed the mechanisms for authentication (from CRAM-MD5 to PLAIN), Outlook Express should have no problem authenticating now. Here's what you need to do:

  1. Start Outlook Express and go to the menu option Tools->Accounts.
  2. When the new window opens, click on the "Mail" tab, then click on the "Add" button followed by "Mail". You will be presented with a wizard, enter the data as you see fit until you reach the 3rd page where you're asked for your Sending and Receiving servers. Please enter the IP address of your server (in my example it's 192.168.2.2) in both text fields and use the pull down to select "IMAP".
  3. Next you'll be asked for your IMAP server's username/password. Please enter the information you found in chart 6 under the "Local IMAP" heading.
  4. The wizard will finish, but you must open up the properties of that connection again. Under the "Mail" tab you should see your connection, highlight it with a single click and select the "Properties" button off to the right. A new window will open
  5. Click on the "Advanced" tab and check the checkboxes so that you are enabling SSL for both sending and receiving. You'll notice the port for your IMAP server has changed to 993.
  6. Click on the "IMAP" tab and under the Root folder path option, enter the word "INBOX" and make sure that "Check for new messages in all folders" is enabled.
  7. Click on the "Servers" tab and enable the box that says "My server requires authentication", then click on the "Settings..." button.
  8. You'll presented with a new window with radio buttons. Click on the option that says "Log on using", thus enabling the text fields below and enter your account information found in chart 6 under the "Local SMTP" heading.

Click the Apply/OK button and you're all set. Check your email and send out a test email to yourself.

Mozilla Thunderbird

Mozilla Thunderbird is becoming a popular email client for both Linux and Windows. The support for IMAP servers is pretty good although there are a large number of unfixed bugs in Mozilla Bugzilla related to IMAP support. Sending mail using the setup described in this HOWTO does not seem to work with Mozilla Thunderbird 1.0.

Note: If you want to send email with Thunderbird, try to manually import the certificate (/etc/ssl/postfix/server.crt). Go to Tools->Options->Advanced->Manage Certificates, click on Import and select server.crt.

Note 2: I've got the setup working with Thunderbird 2.0 (2.0.0.6 to be exact). In the "Server Settings" page under "Security Settings" section click on the SSL radio button and make sure the "Use secure authentication" tickbox is clear.

Squirrelmail Webmail Setup

We're going to set up Squirrelmail Webmail. Although this step is optional, it may be of use to some. If you're a fan of webmail, this piece of software is a godsend. Let's continue.

I will assume you have a working Apache and PHP setup. First thing to do is SSH into your server again and become root.

see SquirrelMail for installation instructions

Bogofilter Mail Filtering Solution

see bogofilter

SpamAssassin Mail Filtering Solution

By Proteus

I have managed to get SpamAssassin 2.55-r1 - this version has bayesian filtering, too.

I implemented it in a very simple way (basically combining the .procmailrc file from this guide and the example file that comes with SA, setting up a .spam maildir and setting up cronjobs to let SA learn the difference between spam and other emails):

Emerge SpamAssassin

First thing we do is emerge the program. It has a few perl dependencies, but shouldn't take that long.

root@server # emerge spamassassin

You can emerge new versions of SpamAssassin rulesets, when they are released by the authors. It adds a cronjob to the system that daily fetches the new rulesets.

root@server # echo "mail-filter/spamassassin-ruledujour ~x86" >> /etc/portage/package.keywords
root@server # emerge spamassassin-ruledujour
root@server # chmod +x /etc/cron.daily/rulesdujour

Edit your .procmailrc File

Open up your .procmailrc file which is located in your user's home directory. You will need to add the following:

File: ~/.procmailrc
 #set up a Spam maildir where all the spam goes for teaching SA spam vs. non-spam
 #and to be sure that no mail - even if detected as spam - gets lost (like when you pipe it to /dev/null)

 SPAM_FOLDER= $MAILDIR/.spam/

 #pipe mails through SA (this is basically from the example files
 #but I use a higher limit, every mail up to 512 kB is filtered)
 #spamc is the client programm for the daemonized
 #version of SA (designed to keep load and overhead down)
 #If you don't run SA as a daemon change "spamc" to "/usr/bin/spamassassin"
 #If you do use spamc here you must add spamd to your runlevel
 #like this: rc-update add spamd default

 :0fw: spamassassin.lock
 * < 524288
 | spamc

 #All mail tagged as spam (eg. with a score higher than the set threshold)
 #is moved to ".spam".

 :0:
 * ^X-Spam-Status: Yes
 $SPAM_FOLDER

 #Work around procmail bug: any output on stderr will cause the "F" in
 #"From" to be dropped.  This will re-add it.
 #(This is taken directly from the SA example file)

 :0
 * ^^rom[ ]
 {
   LOG="*** Dropped F off From_ header! Fixing up. "

   :0 fhw
   | sed -e '1s/^/F/'
 }

Try your best to leave the rest of the file as it is described above.

Setup Spam Maildir

user@server # maildirmake -f spam ~/.maildir

Configure SpamAssassin

This can be done automatically (almost) by using a script you can find here: [1] Place the config file here: /etc/mail/spamassassin/local.cf

If you setup SA with bayesian scanning enabled you must teach it to detect spam first.

This is done by putting all detected spam in the .spam maildir (when some spam gets through, put it there manually, so SA can adapt) and then letting SA learn from those mails and from those mails (considered good) in your .inbox.

You can do this by hand or - as I did - use a cronjob to do it.

SA will only start to use the bayesian scan after learning from at least 200 mails.

If you only use SA in standard mode or just merge the "stable" version (i.e. without using ACCEPT_KEYWORD="~x86") you do not need to do the next steps. The current stable version is 2.44 as of this writing and does not contain bayesian filtering at all... (As it seems you can add bogofilter for this task instead, but I have no clue about that, yet.)

Setup Cronjob for sa-learn (bayesian filter teaching program)

complex

Here is a script for sa-learn to do the job for IMAP,
you can use it for a personal use or make it run as a daily cron job,
it scan for folder startwith Ham & Spam and their subfolder,
so it simply learning Ham with Ham, Spam with Spam.

If you wanna use it as a daily cron job,
copy this code, put it under /etc/cron.daily and make it executable.

If you wanna use this for personal use,
copy this code, put it like the bogofilter script example above,
mention the different between this code with bogofilter one,
and the permission problem with spamassassin DB.

Code: satrainer.py
#!/usr/bin/python

#configuration
sa_learn = "/usr/bin/sa-learn"
niceValue = 0

mailDir = ".maildir"
dbDir = ".spamassassin"

hamDir = ".Ham"
spamDir = ".Spam"

import os, pwd, subprocess
os.nice(niceValue)

def userMailWalker(curUserPWdb):
    #tmpEUID = os.geteuid();
    #os.seteuid(curUserPWdb[2])
    curUser = curUserPWdb[0]
    curUserDir = curUserPWdb[5]
    curUserMailDir = os.path.join(curUserDir, mailDir)
    curUserDbDir = os.path.join(curUserDir, dbDir)
    if os.path.exists(curUserMailDir) is True:
        userMailDirList = os.listdir(curUserMailDir)
        for curMailDir in userMailDirList:
            curMailDirPath = os.path.join(curUserMailDir, curMailDir, "cur")
            print curMailDirPath
            if curMailDir.startswith(hamDir):
                args = ['--no-sync','--username='+curUser, '--dbpath', curUserDbDir, '--ham', curMailDirPath]
                subprocess.Popen([sa_learn]+args)
            elif curMailDir.startswith(spamDir):
                args = ['--no-sync','--username='+curUser, '--dbpath', curUserDbDir, '--spam', curMailDirPath]
                subprocess.Popen([sa_learn]+args)
        subprocess.Popen([sa_learn]+["--sync"])
    #os.seteuid(tmpEUID)

if cmp(pwd.getpwuid(os.getuid())[0], "root") is 0:
    #it's root, maybe this's a cron job
    for curPWdb in pwd.getpwall():
        os.seteuid(pwd.getpwnam('root')[2])
        userMailWalker(curPWdb)
else:
    userMailWalker(pwd.getpwuid(os.getuid()))

simple

Please enter the following into your crontab. In the code block below, make sure you substitute the home directory with one more appropriate to your server. For instance, mine would read: /home/beowulf/.maildir/.spam - yours will be different.

Code: Crontab
 #This scans for spam and for good mails every half hour.
 #Set the interval (30 minutes) appropriatly for your convenience and the amount of mails you get.

 */30 * * * *    sa-learn --dir --spam /home/user/.maildir/.spam > /dev/null 2>&1
 */30 * * * *    sa-learn --dir --ham  /home/user/.maildir/ > /dev/null 2>&1

Conclusion and Testing

So, I hope I haven't left out anything but I think this is all needed to enable spam-filtering with SpamAssassin.

You can check whether or not an email has been scanned by looking at the mail headers, there should be some looking similar to those when it has been scanned:

Code: Spam Mail Header
X-Spam-Status: No, hits=2.1 required=5.0
  tests=HTML_00_10,HTML_MESSAGE,NO_REAL_NAME
  version=2.55
X-Spam-Level: **
X-Spam-Checker-Version: SpamAssassin 2.55 (1.174.2.19-2003-05-19-exp)

Troubleshooting

So something went wrong.... Can't figure out what it is? This section will increase as problems arise with the new version2 of this guide.

Version?

As I have learned from the beginning of this guide, software updates can make a guide worthless. You'll notice in section 1.3 I list the software versions I have tested this set up on. When software updates, it causes problems. One such problem was SASL which changed the way the CMUSecret worked. In short, check and see if a different version of software is the reason why something isn't working

Hostname Problems

It seems different software reads the hostname from different places. Please make sure that your /etc/hosts file is up to date and holds the appropriate lines. Here's mine for comparison:

File: /etc/hosts
127.0.0.1       localhost
192.168.2.2     Chimera.apparition.ath.cx               Chimera
192.168.2.3     Illusion.apparition.ath.cx              Illusion

Another possibility is to check these files:

root@server # cat /etc/hostname && cat /etc/dnsdomainname && cat /etc/nisdomainname
root@server # rc-update add domainname default
root@server # /etc/init.d/domainname start

We've started a service that will set your domainname based on what is held in those three files. You can test what your FQDN is by entering hostname -f

Checking Your Logs

Most logging facilities offered by Gentoo log by default all mail error/info/warnings. If not, read up on how to set it up so it does as the logging is priceless. I recommend reading the Gentoo Security Guide found by clicking here

Restarting Servers

Although it may seem obvious, occassionaly people do forget to restart their servers after making changes. Not only that, but occasionally a restart isn't enough and you must action stop the server, then start it over again. I ran into problems using the command "# postfix reload" where it would not re-read my config file changes. I had to stop the server, then restart it. Weird eh?

Rechecking Config Files

Occassionaly after looking at a very large page of text, your eyes jump lines, occassionaly reading the same line more than once.... or is that just me? In any case, I've tried to make the config file setups as detailed as possible. If nothing is working, and you can't figure it out, double check your config files, see if they match mine, and where they differ, it should only be for personal reasons (IE: network, username/pass, etc).

Folders Not Showing Up In Email Client

One problem that may exist is your subfolders which you have redirected email to using procmail are not showing up in your email client. One option to look for is to subscribe to all folders. Many email clients, even SquirrelMail require the user to specify which subfolders to subscribe to before they show the email there. Also, keep in mind that the subfolders will be created by procmail when email arrives. There's no need to maildirmake any directories aside from the main one (~/.maildir).

Another place to look is the file ~/.maildir/courierimapsubscribed. If you open this file in a text editor you'll notice it has a list of your subsribed folders. Simply add any folders you want in this file. One such example:

Code: ~/.maildir/courierimapsubscribed
INBOX.gentoo
INBOX.gentoo-gwn
INBOX.gentoo-announce

Normally your email client will take care of this, but this is another place to look for errors. Simply restart your courier-imapd-ssl server after editing this file and you're set. Thanks to Fragbeestje for bringing this to our attention.

Can't Connect to Server

There may be a number of reasons why your email client cannot connect to your server's SMTP or IMAP server. Make sure you've opened the necessary ports in your firewall (25 for SMTP and 993 for Imaps). Also, check if you have a ALL:PARANOID in your /etc/hosts.deny. Another place to look is if you're using (x)inetd. If all this fails, netstat -a will show whether or not your server is listening to ports or not. If it isn't, you may have more serious problems than the server not listening, but rather a socket error.

Outlook Spitting Out Errors

Outlook is fickle for lack of a better word. I have it working fine, and I'm confident that the setup I have described will work for you. However, I have been proven wrong on more than one occassion. One reason OE spits out errors is because of Norton Anti-Virus outbound email scanning. If you are having problems sending email, either not connecting or errors in your logs, try disabling outbound scanning. It's a known problem that can be researched on Google.

Resources

I used many resources found from all over the net in my attempts to create this system. Hopefully i have left nothing out and perhaps these links can serve as a starting point for your email endeavours.

Acknowledgements

A special thanks to both Proteus and puddpunk for allowing me to present their work on the front page of this thread. Their Spam solutions has helped flesh out this guide in becoming a single complete setup in a home email system. I tip my hat to you guys

My thanks to many people for their guides as I have used there knowledge in assembling this one. With help from the manual pages, the people replying in this thread (too many to list) and these resources:

Concerns or Compliments? Please use the Discussion section.

Retrieved from "http://www.gentoo-wiki.info/HOWTO_Email_System_for_the_Home_Network_Part_2"

Last modified: Sat, 28 Jun 2008 12:46:00 +0000 Hits: 48,898