Installing OpenSSL and OpenSSH

This document describes how to install OpenSSH (Secure Shell) and OpenSSL (Secure Sockets Layer) on Linux. SSL is a general security library that provides encryption for Web services, VPNs, and other communication software. SSH is a secure replacement for telnet and ftp.

OpenSSL

Although the standard OpenSSL installation will work for some purposes, if you plan to install OpenSSH you need to install OpenSSL twice, once for the static libraries and once to get the shared ones. Otherwise, the OpenSSH configure script may complain about headers not matching the library or OpenSSL headers not being found. It's important to specify the paths when compiling OpenSSL to avoid these errors so you are certain where everything is being installed.

First, install libz, which can be downloaded from the OpenSSL site, if you don't already have it.

Second, install openssl:

./config --prefix=/usr/local --openssldir=/usr/local/ssl make make test make install (as root) ./config shared --prefix=/usr/local --openssldir=/usr/local/ssl make clean make make install (as root) cd /usr/local/ssl/lib cp * /usr/lib
To avoid getting the following error later when you compile OpenSSH:

configure: error: Your OpenSSL headers do not match your library
copy all the SSL include files everywhere:

cd /home/tjnelson/openssl/openssl-* cd include/openssl cp * /usr/include cp * /usr/local/ssl/include cp * /usr/local/ssl/include/openssl
and then add /usr/local/ssl/lib to /etc/ld.so.conf and type

lcdonfig
(as root). This is essential in avoiding "header not found" and "headers do not match your library" errors when you go trying to make OpenSSH.

If the error persists, check config.log to find out what OpenSSH thinks is happening. Sometimes, the actual error has nothing to do with the error message. Or try these commands:

cd /home/tjnelson/openssl/openssl-*/include/openssl cp * /usr/local/ssl/include cd /home/tjnelson/openssl/openssl-* cp lib* /usr/local/ssl/lib/ cp lib* /usr/lib/ ldconfig cd /home/tjnelson/openssl/openssl-*/include/openssl cp * /usr/include/ cp * /usr/local/ssl/include/ cp * /usr/local/ssl/include/openssl

Sometimes, however, libssl complains about something called "pod2man" and refuses to install:

No working pod2man found. Consider installing a new version. As a workaround, we'll use a bundled old copy of pod2man.pl. sh: util/pod2man.pl: No such file or directory
In that case, you must install it manually:

cp libssl.a /usr/local/ssl/lib/ cp *.a /usr/local/ssl/lib/ cp *.so* /usr/local/ssl/lib/ cd include/openssl mkdir /usr/include/openssl/ cp * /usr/include/openssl/ cp * /usr/local/ssl/include/openssl/

In many cases, if you upgrade OpenSSL you also have to rebuild OpenSSH, or you will get a message like:

Starting SSH daemon OpenSSL version mismatch. Built against 907002, you have 90703f startproc: exit status of /usr/sbin/sshd: 255

See below for the procedure.

OpenSSH

OpenSSH requires a working installation of SSL. If OpenSSL is not already installed, install it as described above before proceeding. For OpenSSH version 3.4 or above, it is necessary to create a new user named "sshd" and give a safe place to chroot to. Depending on your system, you may also need to install TCP Wrappers and zlib.

Preparation

Add 'sshd' user: If you haven't already done so, create the new user.

mkdir /var/empty chown root:sys /var/empty chmod 700 /var/empty groupadd sshd useradd -g sshd -c 'sshd privsep' -d /var/empty \ -s /bin/false sshd

zlib: On some systems, it's necessary to upgrade zlib before OpenSSH will compile.
PAM (not recommended): Use of PAM is no longer recommended. If you want to use PAM, the PAM headers and the "--with-pam" configuration option are needed. DO NOT INSTALL PAM unless you are sure of what you're doing! Carelessly installing PAM could trash your system! Make sure your system really uses PAM before using this option. If you need PAM to install ssh, a safe precaution is to install the headers only, and copy the libraries and header files manually into the ssh directory.
PAM_TTY_KLUDGE used to be required, but is no longer necessary, because it is now defined automatically in linux.

Install TCP Wrappers

If your system uses tcpd, and you don't have libwrap, install Wietse Venema's IPv4 tcp-wrappers . There are several versions of tcpd. Two are from porcupine.org, and one of these is for IPv6. Do not install the tcp-wrappers for IPv6 or tcpd from other locations. They will not work. (This step is only needed if you compile ssh with TCP Wrappers.)

Unfortunately, libwrap software is quite old and no longer compiles in Linux without modification. It also uses the old /etc/inetd.conf file, instead of /etc/xinetd.d. Normally, this would mean a lot of messy renaming of your internet daemons. But luckily, compiling sshd with libwrap avoids this problem, and sshd can use tcp-wrappers without all that mucking around with tcpd.

Solution: Edit Makefile and set the variable REAL_DAEMON_DIR to match the location of your existing tcpd.
Then edit the file percent_m.c and change

extern char *sys_errlist[];
to

extern __const char *__const sys_errlist[];
Edit the file scaffold.c and remove line 28 where it says

extern char *malloc();
make
make install
If "make install" doesn't work, do the following:
copy tcpd.h to /usr/local/include/
copy tcpd to /usr/sbin/
copy libwrap.a to /usr/local/lib/
make sure /usr/local/lib is in your /etc/ld.so.conf file
ldconfig

Patch OpenSSH to fix hang-on-exit bug (Optional)

Note: this problem seems to have finally been fixed, and it is no longer necessary to patch openssh. This information applies to earlier versions.
The ssh client in OpenSSH hangs if a command is started in background. For example, if you use nohup to start a batch file while you're logged in over ssh, the ssh client will hang when you logout, and must be killed manually. The solution involves editing two files in the OpenSSH source code before installing.

File 1: clientloop.c
In the file clientloop.c near line 1192, in the function client_input_channel_req after the line
success = 1;
add the line
quit_pending=1;
Here is a patch file:
--- clientloop.c.bak Sun Feb 16 14:58:30 2003 +++ clientloop.c Sun Feb 16 14:58:37 2003 @@ -1214,6 +1214,7 @@ } else if (strcmp(rtype, "exit-status") == 0) { success = 1; exit_status = packet_get_int(); +quit_pending=1; packet_check_eom(); } if (reply) {

File 2: session.c
The clientloop.c fix solves some cases of ssh hanging, and allows ssh to exit cleanly for nohup'ed processes. However, in most cases it's also necessary to modify session.c as described by Jani Jaakkola:

--- session.c.bak Sun Feb 16 21:22:20 2003 +++ session.c Sun Feb 16 21:27:44 2003 @@ -1705,6 +1705,21 @@ debug("session_exit_message: session %d channel %d pid %d", s->self, s->chanid, s->pid); + + /* Add hang-on-exit bug workaround */ + if (s->ttyfd != -1 && c->istate == CHAN_INPUT_OPEN) { + fd_set dummy_read, dummy_write; + FD_ZERO(&dummy_read); + FD_SET(s->ttyfd,&dummy_read); + FD_ZERO(&dummy_write); + channel_after_select(&dummy_read,&dummy_write); + verbose("hang-on-exit-workaround applied"); + chan_read_failed(c); + } + + if (WIFEXITED(status)) { channel_request_start(s->chanid, "exit-status", 0); packet_put_int(WEXITSTATUS(status));

Patch #2 - Add delay after unsuccessful login (Optional)

This patch adds a delay after unsuccessful login. This makes it inconvenient for attackers to send dictionary lists of common passwords, because it slows their dictionary attack down by a factor of 1000 or more. (Notice that it will not affect attackers who try only a single password per user).

File 1: auth2.c
In function input_userauth_request(), change:

debug("attempt %d failures %d", authctxt->attempt, authctxt->failures);
to

debug("attempt %d failures %d", authctxt->attempt, authctxt->failures); if(authctxt->failures > 2) sleep(10);

Patch #3 - Add delay before getting password (Optional)

This patch adds a delay before any login, making it inconvenient for attackers to send dictionary lists of common passwords. The idea was from this site.

In file auth-passwd.c, add a sleep() function near the beginning of the auth_passwd() function, right after the declarations, like so:

int auth_password(Authctxt *authctxt, const char *password) { struct passwd * pw = authctxt->pw; int result, ok = authctxt->valid; #if defined(USE_SHADOW) && defined(HAS_SHADOW_EXPIRE) static int expire_checked = 0; #endif /* Password authentication delay */ sleep(4);

Compile and install OpenSSH

In the OpenSSH directory, type

./configure --prefix=/usr
or

./configure --prefix=/usr --with-tcp-wrappers
then

make su make install

Restart sshd, samba, and apache; otherwise these and possibly other applications may crash silently.

cd /etc/rc.d ./apache restart ./sshd restart ./smb restart
Note: sshd sometimes tries to restart before the port is closed, giving "failed" message. In this case, use the commands:

./sshd stop ./sshd start
Edit sshd_config and ssh_config as needed. Be sure to edit the correct sshd_config. This file could be in /etc, /usr/etc/, /etc/ssh, and in a variety of other places. If you're using PAM, edit /etc/pam.d/sshd as well.

Problems

If 'configure' can't find ssl, change the configure command to:

./configure --prefix=/usr --with-ssl-dir=\
/usr/local/ssl --with-tcp-wrappers

Sometimes compilation of openssh bombs out with the following error messages:

sshd.c:1660: error: storage size of req isn't known
sshd.c:1663: warning: implicit declaration of function fromhost
sshd.c:1660: warning: unused variable req

Another frequent problem is that configure bombs out with the message:

configure: error: Your OpenSSL headers do not match your library

The easiest solution is to go to your openssl directory and type:

make install
cd include/openssl
cp * /usr/include/openssl/
cp * /usr/local/ssl/include/openssl/

Compile openssh, using the following command line, and substitute the appropriate path after --with-ssl-dir :

./configure --prefix=/usr --with-tcp-wrappers \
--with-ssl-dir=/usr/local/ssl
make
make install

If it says:

Connecting to carbon...
 OpenSSL version mismatch. Built against 90603f, you have 90607f
 Couldn't read packet: Connection reset by peer

Get rid of your old libcrypto libraries in /usr/lib and rerun ldconfig:

cd /usr/local/ssl/lib
cp * /usr/lib/
ldconfig

Then rebuild and reinstall openssh (make clean; make; make install), and restart sshd.

Sometimes ssh has the following connection problem:
On client

Request for subsystem 'sftp' failed on channel 0
Couldn't read packet: Connection reset by peer

On server

sshd[7079]: Accepted password for tjnelson from 63.127.146.196 port 32777 ssh2
sshd[7079]: subsystem request for sftp
sshd[7079]: error: subsystem: cannot stat /usr/local/libexec/sftp-server: 
No such file or directory
sshd[7079]: subsystem request for sftp failed, subsystem not found

To solve this problem, do the following:
Edit /etc/ssh/sshd_config and add:

Subsystem       sftp    /usr/local/libexec/sftp-server
Protocol 2

Create /usr/libexec and make a link to sftp-server. For some reason sshd sometimes looks there instead of the path specified in its configuration file.

mkdir /usr/libexec
cd /usr/libexec
ln -s /usr/lib/ssh/sftp-server sftp-server

Stop and start sshd (don't restart, because the port takes a while to close).

cd /etc/rc.d
./sshd stop
./sshd restart

If it says:

checking for socklen_t equivalent... configure: 
error: Cannot find a type to use in place of socklen_t

this means your computer is totally screwed, and you should take it out back and shoot it.

If it says:
sshd re-exec requires execution with an absolute path
The startup script needs to be modified to include the full path of sshd.
If it says:
Permission denied (publickey,keyboard-interactive).
This can happen after an upgrade when you change from PAM to password authentication. Edit /etc/ssh/sshd_conf and change PasswordAuthentication to yes.

Warning about sftp:
The sftp program doesn't check whether the file is transferred correctly. If it is not possible to create the file at the remote destination, sftp may silently fail to transfer the file.

Back