Note that this document was written for 10.0 and 10.1. It has been reported to work with 10.2 but it is untested by the author. Note that 10.3 uses postfix as the mailer so none of this article is applicable to 10.3 or 10.4

Note that at ish, we now use the product that we distribute (CommuniGate Pro) for any commercial mail server installations, so we have less experience with Apple’s mail server performance in recent builds of OSX. CommuniGate Pro is extremely fast and stable under OSX.

The problem

After struggling with Apple’s built in mail server for a while it became apparent that the product is lacking in essential features and suffers from some fatal bugs. Although the open relay bug was fixed in OSX 10.1.3, several other problems remain:

• bounce messages are not always processed correctly;
• the mail database corruption is common and difficult to recover from;
• the mail database is in proprietary format making it difficult to use other tools to view or post-process mail;
• AMS doesn’t accept mail from external applications like PHP or log extraction utilities that expect sendmail-like delivery;
• minimal feature set with very little flexibility;
• minimal anti-spam options (like SpamAssassin), no virus scanning options, no way to easily integrate a list server, web mail, content filtering, etc., etc.

The solution

There are several options to improve the situation:

• CommuniGate Pro (terrific application if you have a budget)
• postfix, qmail, etc (many people like and use these packages, but they are less used generally than sendmail)
• struggle on with AppleMailServer (life is too short!)
• sendmail

Most people’s criticism of sendmail is usually of its confusing configuration file called sendmail.cf. This article is designed to solve that problem by providing some simple human readable configuration files. A short script does the work of creating the sendmail.cf file.

There are two ways of getting sendmail to work, either on its own or in conjuction with the AppleMailServer. We have made both work, but only use the first option in production. There are too many problems with AppleMailServer to inspire confidence that it will be reliable even for just local delivery. I strongly recommend Option 2 for ease of setup and ease of fixing problems. Option 1 may not work with 10.1.5 as we have not tested it with that version of OSX.

Basic setup

Before you do anything else, install the developer package which came with OSX. This installs some tools we are going to need. You should also login as root or type “su” to become root.

Let’s start by turning on the tag that causes sendmail to start automatically at system startup. Use your favourite text editor (eg pico, vim, BBEdit) to edit this file:

/etc/hostconfig

Change the line

MAILSERVER=-NO-

to

MAILSERVER=-YES-

Next, we need to create some files that sendmail uses to work out what to do. Change the 10.0.0 below to reflect your own network (eg. 192.169.0 or 10.1). Only the IP addresses and domains listed here will be able to use sendmail to relay mail. Note that 10.0.0 is equivalent to 10.0.0.0/24 or netmask 255.255.255.0. This prevents others from using your mail server to send spam (open relay). See www.ordb.org for more information.

Note that upgrading to OSX 10.1.5 results in Apple’s installer deleting this file! Just recreate it.

Create /etc/mail/access

localhost RELAY
10.0.0 RELAY

Create /etc/mail/aliases. This is fairly obvious - it maps certain addresses to other addresses. Sendmail will use this file as well as the aliases in NetInfo to map addresses, put them wherever is more convenient.

root: admin
support: support@ITcontractor.com.au
widgetsales: fred, jo

You can see how easy it is to create a simple distribution list to both fred and jo.

Create /etc/mail/local-host-names. This tells sendmail what domains it responds to as local. In other words, mail to these domains will be delivered to local users if they exist or bounced if they don’t exist.

yourdomainname.com.au
mail.yourdomain.com.au
anotherdomain.com.au

Create /etc/mail/virtusertable. This file is used to map various domains that sendmail responds to against the local users to whom the mail is delivered.

sales@yourdomain.com.au    widgetsales
kylie@anotherdomain.com.au kylie
@anotherdomain.com.au      error:nouser No such user here

This looks a little complicated but isn’t really. The virtual user file is always parsed first, followed by the aliases file. So, sales@yourdomain.com.au will end up sent to both fred and jo. christina@yourdomain.com.au will be mapped to the local user christina, because there is no explicit rule for her. All users at anotherdomain.com.au (except kylie) will be rejected, even if they match a local user name. More details can be found in the sendmail howto

create /etc/mail/build

chmod go-w / /etc /etc/mail /var /var/spool /var/spool/mqueue
cd /usr/share/sendmail/conf/m4
m4 /etc/mail/sendmail.mc > /etc/mail/sendmail.cf
for map in virtusertable access domaintable mailertable
do
if [ -f /etc/mail/${map} ] ; then
makemap hash /etc/mail/${map} < /etc/mail/${map}
sleep 1
fi
done
newaliases

niutil -create . /locations/sendmail
niutil -createprop . /locations/sendmail sendmail.cf /etc/mail/sendmail.cf
kill -HUP `head -1 /var/run/sendmail.pid`

This last file is of my own creation. It allows us to easily change the config files, rebuild the hash files (which are a preprocessed version of the config files sendmail uses) and restart sendmail.

Next, you need to either configure sendmail to forward local mail to AppleMailServer or to mailboxes in /var/spool/.

1. AppleMailServer
2. mailboxes/qpopper

Change history

• 1 November 2001: original document
• 12 December 2001: fixed some typos and added clarifications.
• 4 March 2002: made line breaks easier to read in config files, added reasons for wanting to use sendmail. Reworked the introduction
• 15 April 2002: updated qpopper link to 4.0.4
• 20 June 2002: updated information for 10.1.5
• 28 June 2002: reworked whole article to make it clearer, added more 10.1.5 information and split into several sections. Added virtual domain configuration.
• 27 October 2003: added note about 10.3 using exim.