Installing OpenSSL and OpenSSH

This document describes how to install OpenSSH (Secure Shell) and OpenSSL (Secure Sockets Layer) on Linux. SSL is a general security library that provides encryption for Web services, VPNs, and other communication software. SSH is a secure replacement for telnet and ftp.


Although the standard OpenSSL installation will work for some purposes, if you plan to install OpenSSH you need to install OpenSSL twice, once for the static libraries and once to get the shared ones. Otherwise, the OpenSSH configure script may complain about headers not matching the library or OpenSSL headers not being found. It's important to specify the paths when compiling OpenSSL to avoid these errors so you are certain where everything is being installed.

First, install libz, which can be downloaded from the OpenSSL site, if you don't already have it.

Second, install openssl:
    ./config --prefix=/usr/local --openssldir=/usr/local/ssl
    make test
    make install  (as root)
    ./config shared --prefix=/usr/local --openssldir=/usr/local/ssl
    make clean
    make install   (as root)
    cd /usr/local/ssl/lib
    cp * /usr/lib
To avoid getting the following error later when you compile OpenSSH:
    configure: error: Your OpenSSL headers do 
    not match your library
copy all the SSL include files everywhere:
    cd /home/tjnelson/openssl/openssl-*   
    cd include/openssl
    cp * /usr/include
    cp * /usr/local/ssl/include
    cp * /usr/local/ssl/include/openssl
and then add /usr/local/ssl/lib to /etc/ and type
(as root). This is essential in avoiding "header not found" and "headers do not match your library" errors when you go trying to make OpenSSH.

If the error persists, check config.log to find out what OpenSSH thinks is happening. Sometimes, the actual error has nothing to do with the error message. Or try these commands:
    cd /home/tjnelson/openssl/openssl-*/include/openssl
    cp * /usr/local/ssl/include
    cd /home/tjnelson/openssl/openssl-*
    cp lib* /usr/local/ssl/lib/
    cp lib* /usr/lib/
    cd /home/tjnelson/openssl/openssl-*/include/openssl 
    cp * /usr/include/
    cp * /usr/local/ssl/include/
    cp * /usr/local/ssl/include/openssl

Sometimes, however, libssl complains about something called "pod2man" and refuses to install:
   No working pod2man found.  Consider installing a new version.
   As a workaround, we'll use a bundled old copy of
   sh: util/ No such file or directory
In that case, you must install it manually:
    cp libssl.a /usr/local/ssl/lib/
    cp *.a /usr/local/ssl/lib/
    cp *.so* /usr/local/ssl/lib/
    cd include/openssl
    mkdir /usr/include/openssl/
    cp * /usr/include/openssl/
    cp * /usr/local/ssl/include/openssl/

In many cases, if you upgrade OpenSSL you also have to rebuild OpenSSH, or you will get a message like:
Starting SSH daemon
OpenSSL version mismatch. Built against 907002, 
you have 90703f
startproc:  exit status of /usr/sbin/sshd: 255
See below for the procedure.


OpenSSH requires a working installation of SSL. If OpenSSL is not already installed, install it as described above before proceeding. For OpenSSH version 3.4 or above, it is necessary to create a new user named "sshd" and give a safe place to chroot to. Depending on your system, you may also need to install TCP Wrappers and zlib.


  1. Add 'sshd' user: If you haven't already done so, create the new user.
    mkdir /var/empty
    chown root:sys /var/empty
    chmod 700 /var/empty
    groupadd sshd
    useradd -g sshd -c 'sshd privsep' -d /var/empty \
      -s /bin/false sshd

  2. zlib: On some systems, it's necessary to upgrade zlib before OpenSSH will compile.

  3. PAM (not recommended): Use of PAM is no longer recommended. If you want to use PAM, the PAM headers and the "--with-pam" configuration option are needed. DO NOT INSTALL PAM unless you are sure of what you're doing! Carelessly installing PAM could trash your system! Make sure your system really uses PAM before using this option. If you need PAM to install ssh, a safe precaution is to install the headers only, and copy the libraries and header files manually into the ssh directory.

    PAM_TTY_KLUDGE used to be required, but is no longer necessary, because it is now defined automatically in linux.

Install TCP Wrappers

If your system uses tcpd, and you don't have libwrap, install Wietse Venema's IPv4 tcp-wrappers . There are several versions of tcpd. Two are from, and one of these is for IPv6. Do not install the tcp-wrappers for IPv6 or tcpd from other locations. They will not work. (This step is only needed if you compile ssh with TCP Wrappers.)

Unfortunately, libwrap software is quite old and no longer compiles in Linux without modification. It also uses the old /etc/inetd.conf file, instead of /etc/xinetd.d. Normally, this would mean a lot of messy renaming of your internet daemons. But luckily, compiling sshd with libwrap avoids this problem, and sshd can use tcp-wrappers without all that mucking around with tcpd.

Solution: Edit Makefile and set the variable REAL_DAEMON_DIR to match the location of your existing tcpd.
Then edit the file percent_m.c and change
extern char *sys_errlist[];
 extern __const char *__const sys_errlist[];
Edit the file scaffold.c and remove line 28 where it says
 extern char *malloc();
make install
If "make install" doesn't work, do the following:
copy tcpd.h to /usr/local/include/
copy tcpd to /usr/sbin/
copy libwrap.a to /usr/local/lib/
make sure /usr/local/lib is in your /etc/ file

Patch OpenSSH to fix hang-on-exit bug (Optional)

Note: this problem seems to have finally been fixed, and it is no longer necessary to patch openssh. This information applies to earlier versions.
The ssh client in OpenSSH hangs if a command is started in background. For example, if you use nohup to start a batch file while you're logged in over ssh, the ssh client will hang when you logout, and must be killed manually. The solution involves editing two files in the OpenSSH source code before installing.

File 1: clientloop.c
In the file clientloop.c near line 1192, in the function client_input_channel_req after the line
     success = 1;
add the line
Here is a patch file:
--- clientloop.c.bak    Sun Feb 16 14:58:30 2003
+++ clientloop.c        Sun Feb 16 14:58:37 2003
@@ -1214,6 +1214,7 @@
        } else if (strcmp(rtype, "exit-status") == 0) {
                success = 1;
                exit_status = packet_get_int();
        if (reply) { 

File 2: session.c
The clientloop.c fix solves some cases of ssh hanging, and allows ssh to exit cleanly for nohup'ed processes. However, in most cases it's also necessary to modify session.c as described by Jani Jaakkola:

--- session.c.bak       Sun Feb 16 21:22:20 2003
+++ session.c   Sun Feb 16 21:27:44 2003
@@ -1705,6 +1705,21 @@
        debug("session_exit_message: session %d channel %d pid %d",
            s->self, s->chanid, s->pid);
+       /* Add hang-on-exit bug workaround */
+        if (s->ttyfd != -1 && c->istate == CHAN_INPUT_OPEN) {
+                fd_set dummy_read, dummy_write;
+                FD_ZERO(&dummy_read);
+                FD_SET(s->ttyfd,&dummy_read);
+                FD_ZERO(&dummy_write);
+                channel_after_select(&dummy_read,&dummy_write);
+                verbose("hang-on-exit-workaround applied");
+                chan_read_failed(c);
+        }
        if (WIFEXITED(status)) {
                channel_request_start(s->chanid, "exit-status", 0);

Patch #2 - Add delay after unsuccessful login (Optional)

This patch adds a delay after unsuccessful login. This makes it inconvenient for attackers to send dictionary lists of common passwords, because it slows their dictionary attack down by a factor of 1000 or more. (Notice that it will not affect attackers who try only a single password per user).

File 1: auth2.c
In function input_userauth_request(), change:
debug("attempt %d failures %d", authctxt->attempt, authctxt->failures);
debug("attempt %d failures %d", authctxt->attempt, authctxt->failures);
if(authctxt->failures > 2) sleep(10);

Patch #3 - Add delay before getting password (Optional)

This patch adds a delay before any login, making it inconvenient for attackers to send dictionary lists of common passwords. The idea was from this site.

In file auth-passwd.c, add a sleep() function near the beginning of the auth_passwd() function, right after the declarations, like so:
auth_password(Authctxt *authctxt, const char *password)
        struct passwd * pw = authctxt->pw;
        int result, ok = authctxt->valid;
#if defined(USE_SHADOW) && defined(HAS_SHADOW_EXPIRE)
        static int expire_checked = 0;

/* Password authentication delay */

Compile and install OpenSSH

In the OpenSSH directory, type
./configure --prefix=/usr 
./configure --prefix=/usr --with-tcp-wrappers
make install

Restart sshd, samba, and apache; otherwise these and possibly other applications may crash silently.
cd /etc/rc.d
./apache restart
./sshd restart 
./smb restart
Note: sshd sometimes tries to restart before the port is closed, giving "failed" message. In this case, use the commands:
./sshd stop
./sshd start
Edit sshd_config and ssh_config as needed. Be sure to edit the correct sshd_config. This file could be in /etc, /usr/etc/, /etc/ssh, and in a variety of other places. If you're using PAM, edit /etc/pam.d/sshd as well.


  1. If 'configure' can't find ssl, change the configure command to:
    ./configure --prefix=/usr --with-ssl-dir=\
    /usr/local/ssl --with-tcp-wrappers

  2. Sometimes compilation of openssh bombs out with the following error messages:
    sshd.c:1660: error: storage size of req isn't known
    sshd.c:1663: warning: implicit declaration of function fromhost
    sshd.c:1660: warning: unused variable req
  3. Another frequent problem is that configure bombs out with the message:
    configure: error: Your OpenSSL headers do not match your library
    The easiest solution is to go to your openssl directory and type:
    make install
    cd include/openssl
    cp * /usr/include/openssl/
    cp * /usr/local/ssl/include/openssl/
    Compile openssh, using the following command line, and substitute the appropriate path after --with-ssl-dir :
    ./configure --prefix=/usr --with-tcp-wrappers \
    make install
  4. If it says:
    Connecting to carbon...
     OpenSSL version mismatch. Built against 90603f, you have 90607f
     Couldn't read packet: Connection reset by peer
    Get rid of your old libcrypto libraries in /usr/lib and rerun ldconfig:
    cd /usr/local/ssl/lib
    cp * /usr/lib/
    Then rebuild and reinstall openssh (make clean; make; make install), and restart sshd.
  5. Sometimes ssh has the following connection problem:
    On client
    Request for subsystem 'sftp' failed on channel 0
    Couldn't read packet: Connection reset by peer
    On server
    sshd[7079]: Accepted password for tjnelson from port 32777 ssh2
    sshd[7079]: subsystem request for sftp
    sshd[7079]: error: subsystem: cannot stat /usr/local/libexec/sftp-server: 
    No such file or directory
    sshd[7079]: subsystem request for sftp failed, subsystem not found
    To solve this problem, do the following:
    Edit /etc/ssh/sshd_config and add:
    Subsystem       sftp    /usr/local/libexec/sftp-server
    Protocol 2
    Create /usr/libexec and make a link to sftp-server. For some reason sshd sometimes looks there instead of the path specified in its configuration file.
    mkdir /usr/libexec
    cd /usr/libexec
    ln -s /usr/lib/ssh/sftp-server sftp-server
    Stop and start sshd (don't restart, because the port takes a while to close).
    cd /etc/rc.d
    ./sshd stop
    ./sshd restart
  6. If it says:
    checking for socklen_t equivalent... configure: 
    error: Cannot find a type to use in place of socklen_t
    this means your computer is totally screwed, and you should take it out back and shoot it.
  7. If it says:
    sshd re-exec requires execution with an absolute path
    The startup script needs to be modified to include the full path of sshd.
  8. If it says:
    Permission denied (publickey,keyboard-interactive).
    This can happen after an upgrade when you change from PAM to password authentication. Edit /etc/ssh/sshd_conf and change PasswordAuthentication to yes.

Warning about sftp:
The sftp program doesn't check whether the file is transferred correctly. If it is not possible to create the file at the remote destination, sftp may silently fail to transfer the file.