Installing sendmail with aliases and procmail support

While the default sendmail that's included in most Linux distributions usually works satisfactorily for an individual workstation user, it's almost always necessary to build and install a better one manually in a production environment. That's because users could be sending mail through Outlook, Thunderbird, PC-Pine, or any number of other email clients.

This page describes how to compile and install sendmail with email aliases and procmail support. Email aliases let you create mailing lists and give alternate names or addresses for users. Procmail is a tool for filtering email to remove spam, sort mail, etc. The procedure for configuring procmail is described here and here.

If you are running a version sendmail earlier than 8.13.6, you need to upgrade because of a serious security problem. As of version 8.12, you must also create a new user (such as smmsp) and change the permissions of various files as described in the file sendmail/SECURITY.


  1. Back up the following files:
  2. Create a user and group for sendmail. This can be anything, but most people choose `mail' or `smmsp'. Many sendmail problems are caused by various files having the wrong ownership or permissions.
    useradd smmsp
    groupadd smmsp
    This step is essential, as sendmail will not start up unless it has its own userid.
    The entries in /etc/passwd and /etc/group should look something like this:
    cat /etc/passwd | grep smmsp
    cat /etc/group | grep ^smmsp

Install libdb

If you don't care about email aliases, skip this step. Otherwise, if you don't have libdb on your system ('locate libdb'), get it from and install it. For Unix/Posix systems:
cd db-4.2.52/build_unix
make install
Make a note of where the libraries were installed (e.g., /usr/local/BerkeleyDB.4.2/lib).

After installing, it may be necessary to perform these three extra steps (as root), depending on whether an old version of libdb is already present.

  1. First, copy db.h to /usr/include. This is essential, as the db.h must match the library. If it doesn't, sendmail won't compile.
    cd db-4.2.52
    cp build_unix/db.h /usr/include/ 

  2. Second, get rid of the old libdb libraries.
    cd /usr/lib
    mkdir libdb-old
    mv libdb* libdb-old

  3. Third, add the new libdb path (/usr/local/BerkeleyDB.4.2/lib) to /etc/ and reconfigure the run-time linker.
    vi /etc/

Compile and install sendmail

Compile sendmail. In some linux distributions, you will need to have the source code for a kernel installed before sendmail will compile. However, sometimes this can be avoided by creating an empty version.h file.
touch /usr/include/linux/version.h
tar -xzvf sendmail.8.11.6.tar.gz
cd send*

  1. First, edit the config files if you plan to use procmail.
    cd cf/cf
    Edit and add
    at the end. Add any other commands as needed.
  2. Create and install new config files using the following command:
    sh Build  
    sh Build  
    make install-cf
    In earlier versions, the file was not copied automatically if an old version of the file already existed. If you try to use an old version of, when you start sendmail it will say:
    NOQUEUE: SYSERR(root): can not chdir(/var/spool/clientmqueue/):
    Permission denied
    This error can also occur if sendmail is running as the wrong user or group.

    Alternatively, install the config files manually.
    cd cf/cf
    cp /etc/mail/
    cp /etc/mail/
  3. Create a devtools/Site/site.config.m4 file containing the paths for libdb. Change the paths to indicate wherever the libdb libraries and include files are located.
    APPENDDEF(`confINCDIRS', `-I/usr/include/db2')
    APPENDDEF(`confLIBDIRS', `-L/usr/lib')
    APPENDDEF(`confLIBS', `-ldb') 
    The exact syntax will vary depending on your system. On one of our computers, where we installed a new libdb, the file contained the following four lines:
    APPENDDEF(`confLIBS', `-lssl -lcrypto -ldb')
    APPENDDEF(`confLIBDIRS', `-L/usr/local/ssl/lib -L/usr/local/ssl/lib
              -L/usr/local/BerkeleyDB.4.2/lib -L/usr/lib')
    APPENDDEF(`confINCDIRS', `-I/usr/local/ssl/include -I/usr/include')
    APPENDDEF(`confMAPDEF', `-DNEWDB')dnl 
    Sometimes compilation fails with undefined references to `pthread_mutex_trylock' and other pthread functions. If this happens, change the confLIBS line to
    APPENDDEF(`confLIBS', `-ldb -lpthread') 

    NOTE: If you use the site.config.m4.example file that is included with sendmail, be sure to comment out items you don't want, or sendmail will probably not compile.

    See more uses of APPENDDEFs in "Thunderbird and APPENDDEFs" below.
  4. Compile sendmail
    sh Build 
    If you change any configuration files and need to compile a second time, use the -c option.
    sh Build -c 
    The -c option rebuilds all the files to include your changes. It should automatically add the correct options to the commands, for example:
    cc -O2 -I. -I../../sendmail   -I../../include  \
         -I/usr/include/db2 -DNEWDB -DNOT_SENDMAIL     \
         -c -o vacation.o vacation.c 

    IMPORTANT: It's necessary to watch the screen continuously while sendmail is building, because the build script does not stop if there's an error.

    If your linux system is really old, you will get error messages like
    map.c:2084: `DB_FCNTL_LOCKING' undeclared (first use in this function)
    This means you need to install a newer copy of the Berkeley DB libraries from
  5. Check to make sure it created an executable sendmail in the directory obj.Linux.#.#.##.i686/sendmail. If it bombs out, edit the file include/libsmdb/smdb.h, removing the lines
    # ifndef NDBM
    # ifndef NEWDB
    ERROR   NDBM or NEWDB must be defined.
    #  endif /* ! NEWDB */
    # endif /* ! NDBM */ 
    and type sh Build -c again.
  6. Install the new sendmail
    sh Build install     
    Scroll back through the output and make sure there were no errors. If it says:
    install: invalid group `smmsp'
    make[1]: *** [install-sendmail] Error 1
    you must fix the groups and repeat.
  7. Set the correct permissions and ownerships:
    mkdir /var/spool/clientmqueue
    chown smmsp:smmsp /var/spool/clientmqueue
    chmod 770 /var/spool/clientmqueue
    chmod 700 /var/spool/mqueue
    chown root.root /var/spool/mqueue
    chmod 777       /var/spool/mail
    chown root.root /var/spool/mail
    chmod 0555       /usr/sbin/sendmail
    chown root.smmsp /usr/sbin/sendmail
    NOTE: /var/spool/mail must be world read/write, otherwise clients can't open their inbox. The permissions for individual mailboxes for each user should be 600:
    -rw-------    1 daboss users     20219537754 2006-03-31 10:15 daboss

    Check to make sure of the following:
    -r-xr-sr-x   root    smmsp  /usr/sbin/sendmail       (as installed)
    -r-xr-xr-x   root    smmsp  /usr/sbin/sendmail       (sometimes necessary)
    drwxrwx---   smmsp   smmsp  /var/spool/clientmqueue  (as installed)
    drwxrwxrwx   smmsp   smmsp  /var/spool/clientmqueue  (sometimes necessary)
    drwx------   root    root   /var/spool/mqueue
    -r--r--r--   root    bin    /etc/mail/
    -r--r--r--   root    bin    /etc/mail/
    (1) The sendmail installation script sets the sendmail binary guid (r-xr-sr-x). With sendmail version 8.14, this now causes email clients such as pine to hang, regardless of the ownership of clientmqueue and mqueue. So we have found it necessary to set sendmail's permissions to 555 (chmod g-s sendmail) until we figure out what's happening.

    (2) Sendmail will always complain about the permissions on clientmqueue unless they are set to 0770. However, at least on Suse systems, this prevents ordinary users from sending mail. The characteristic symptom is that only root can send mail, while all others get an error message.

    (3) The outgoing queue file /var/spool/mqueue must be owned by root:root or sendmail will complain about a bogus queue file and throw away the message.

  8. Add a relay-domains file in /etc/mail.
    This is a list of IP addresses or domain names that are permitted to relay mail, i.e. send mail to port 25 on your system. It should include the computers or domain names on your site, one per line. Example:
  9. Add a local-host-names file in /etc/mail.
    cp /etc/ /etc/mail/local-host-names
    If local-host-names doesn't exist, create one. It should contain a single line: 
    If the server has more than one name, add them to this file, one per line.

Thunderbird and APPENDDEFs

As mentioned above, an APPENDDEF is a compilation flag that may be needed to add functionality. For example, if you use the stock sendmail, Thunderbird users may have trouble sending mail. They will see the following error message:
The message could not be sent because connecting to SMTP server 
{your server name} failed. The server may be unavailable or is refusing SMTP 
connections. Please verify that your SMTP server setting is correct and try 
again, or else contact your network administrator.
In Linux, add the following to sendmail-8.14.4/devtools/Site/site.config.m4:
APPENDDEF(`conf_sendmail_LIBS', `-lsasl')
The main docs at have instructions for other operating systems. See more uses of APPENDDEFs in Step 3 of "Compile and install sendmail" above.

We didn't have to use this option. Every time we re-install a new version of the OS, which contains the stock sendmail, Thunderbird users complain that they can't send mail. What appears to be happening is that T-bird is connecting to port 587 instead of 25 and sending mail by SMTP instead of using IMAP as specified in the account settings. For whatever reason, the stock sendmail never works, but when we compile a custom sendmail as described here the problem goes away.

Compilation problems

Sendmail usually compiles on Linux without a problem as long as libdb is set up. However, sometimes compilation bombs out with a slew of undefined references to do __dn_skipname and related stuff in domain.o. The solution is to add -lresolv to LIBS. Even the most recent version (sendmail-8.14.7 as of this writing) has this problem.

Solution: Edit sendmail-8.14.4/devtools/Site/site.config.m4 and add -lresolv to the confLIBS Appenddef like so:
APPENDDEF(`confLIBS', `-lssl -lcrypto -ldb -lresolv')
APPENDDEF(`confLIBDIRS', `-L/usr/local/ssl/lib -L/usr/local/ssl/lib\
 -L/usr/local/BerkeleyDB.4.2/lib -L/usr/lib')
APPENDDEF(`confINCDIRS', `-I/usr/local/ssl/include -I/usr/include')
APPENDDEF(`conf_sendmail_ENVDEF', `-DMILTER')

Then rebuild sendmail with sh Build -c.

Testing sendmail

  1. First, as root, test it to make sure the 'newaliases' command works. If it says
    Cannot rebuild aliases: no database format defined
    Cannot create database for alias file /etc/mail/aliases 
    It means sendmail couldn't find the libdb files. Check that db.h is in /usr/include, that the new libraries are in their correct location, the old ones have been removed, and that /etc/ has a line indicating the path to the new libraries.
  2. Restart inetd to make sure imap and pop3 aren't crashing.
  3. Make sure sendmail is running. Unlike earlier versions, it sometimes now has two or more entries:
    ps -aux | grep mail
    root   Mar30   0:01 sendmail: accepting connections                 
    smmsp  Mar30   0:00 sendmail: Queue control                         
    smmsp  Mar30   0:01 sendmail: running queue: /var/spool/clientmqueue
  4. Test to make sure users can send and receive mail in all of the following:
    1. smtp client (e.g. pine)
    2. http (e.g. squirrelmail via browser)
    3. pop3 client (e.g. windows)
    4. imap4 client (e.g. windows)
    Also, check the system logs to make sure there are no weird error messages.
  5. If you like pain and you want to change, the easiest way is to edit and then create it using
    m4 /etc/mail/ > /etc/mail/
  6. You may wish to test to ensure that your sendmail installation is not acting as an open mail relay by typing the command: telnet There is also a web service at You can download software to test it yourself from To be valid, the testing must be done using an account on a machine in some other domain (such as a temporary email account on

    However, testing this way is risky. If the testing software discovers that you're an open relay, your hostname or even your entire domain may end up in their public blacklist, thereby blocking valid email sent by your users. Last time I checked, the testing site does this. We have found these blacklists are so inaccurate that we had to disable the blacklist check in spamassassin to avoid losing incoming mail. Thus, remote testing may do more harm than good.

Error messages

Problem: SMTP greeting failure: 421 SMTP connection went away!
Solution: Ownership or permissions of sendmail binary are wrong. Set it to 4555. Or, sendmail is not running.

Problem: pthreads errors when building sendmail
/usr/local/BerkeleyDB.4.2/lib/ undefined reference to `pthread_condattr_setpshared'
/usr/local/BerkeleyDB.4.2/lib/ undefined reference to `pthread_mutexattr_destroy'
/usr/local/BerkeleyDB.4.2/lib/ undefined reference to `pthread_mutexattr_setpshared'
/usr/local/BerkeleyDB.4.2/lib/ undefined reference to `pthread_mutexattr_init'
/usr/local/BerkeleyDB.4.2/lib/ undefined reference to `pthread_mutex_trylock'

Solution: Turn off pthreads in sendmail by changing the confLIBS line in the file devtools/Site/site.config.m4 to:
APPENDDEF(`confLIBS', `-ldb -lpthread')
and rebuild with sh Build -c.

Problem: Sendmail won't start
 Initializing SMTP port (sendmail)/etc/mail/ 
    line 430: readcf: option RunAsUser: unknown user smmsp: 
    No such file or directory 

Solution: Create user smmsp and group smmsp and rebuild sendmail.

Problem: newaliases says:
Cannot rebuild aliases: no database format defined
Cannot create datase for alias file /etc/mail/aliases  

Solution: This means sendmail couldn't find libdb. Compile and install libdb as described above, and rebuild sendmail.

Problem: Unable to receive mail
Solution: If using Suse Linux (or possibly other distributions), check /etc/sysconfig/mail and make sure it contains the line

Problem: Error sending: SMTP greeting failure: 421 SMTP connection broken (reply)
Solution: Sendmail can not chdir to /var/spool/clientmqueue/ due to a permission problem.

Problem: Class regex not available
Solution: In the devtools/Site file, create a new file named site.config.m4 containing the line
The sample file also contains declarations for specifying the location of ssl libraries and include files. Rebuild sendmail using sh Build -c after changing this file. Beware: sendmail may have started anyway despite this error message.

Problem: No error messages, logs say "sent" but no inbox created.
Solution: Check for a stray .forward file in user's home directory.

See also Pine email problems.

Tweaking sendmail configuration

  1. Changing hostname: Some computers are on two different domains. To get sendmail to use the second domain in its headers instead of your FQDN, add the following to and
    It may also be necessary to change the settings in your email client. For example, in pine, if you have the line:
    customized-hdrs = Reply-To:
    change it to
  2. Slowing down dictionary attacks and distributed denial of service (DDoS) attacks: (this information is from
    Add the following to
    # block dos attacks
    # no of seconds  connection rate
    FEATURE(`ratecontrol', ,`terminate')dnl
    # no of connections simultaneously
    define(`confCONNECTION_RATE_THROTTLE', `8')dnl
    FEATURE(`conncontrol', ,`terminate')dnl
    FEATURE(`greet_pause', `5000')dnl
    # stop dictionary attacks
    define(`confMAX_RCPTS_PER_MESSAGE', `10')dnl
    WARNING: The above options can seriously slow down outgoing email messages if you send a large attachment to multiple recipients. Test before using. We have stopped using these features, but they might be useful if your mail server is frequently under attack.
  3. Stopping server from sending receipts: Some people try to check up on you by requesting a receipt when the email is delivered. Spammers may also use this to validate your users' email addresses.
    # no receipts on delivery
    define(`confPRIVACY_FLAGS', `noreceipts')dnl
    Check your email client; some clients send receipts as well.
  4. New anti-spam features in sendmail version 8.14:
    The first rule will reject any incoming message from servers which don't have a proper reverse DNS. The second one blocks messages that have an unqualified HELO/EHLO argument, or that uses one of our names instead of the actual client name. The third one blocks senders who have bad MX records. These are all features of spammers.

    MAP_REGEX must be in your site.config.m4 file for the third one to work.

    NOTE: These rules also slow down outgoing emails that have multiple recipients. Use at your own risk. They only work in Sendmail 8.14 or higher.

After changing the .mc files, run sh Build, sh Build, su, make install-cf as before and restart sendmail.

Sample file

This is the file we use on this system. Some of the features are specific for version 8.14.
# no receipts on delivery
define(`confPRIVACY_FLAGS', `noreceipts')dnl

# use other domain name
dnl# FEATURE(always_add_domain)dnl
dnl# FEATURE(masquerade_envelope)dnl

# dkim
INPUT_MAIL_FILTER(`dkim-filter', `S=inet:8891@localhost')

#anti-spam features for sendmail 8.14

VERSIONID(`$Id:,v 8.1 1999/09/24 22:48:05 gshapiro Exp $')
This is the site.config.m4 file we use on this system (entropy).
APPENDDEF(`conf_sendmail_ENVDEF', `-DMILTER')
This is the site.config.m4 file we use on Engram.
APPENDDEF(`confLIBS', `-lssl -lcrypto -ldb')
APPENDDEF(`confLIBDIRS', `-L/usr/local/ssl/lib -L/usr/local/ssl/lib -L/usr/local/BerkeleyDB.4.2/lib -L/usr/lib')
APPENDDEF(`confINCDIRS', `-I/usr/local/ssl/include -I/usr/include')
APPENDDEF(`conf_sendmail_ENVDEF', `-DMILTER')