Protecting SSH against brute force attacks

About this page

This page explains you how to protect SSH service from brute force attacks by inserting environment variable based authentication before starting SSH login shell.

Even if the password for SSH authentication was cracked by brute force attacks, SSH login shell won't be started unless both the environment variable name and its value (which act as the other password) are not cracked.

Conventional anti brute force attack methods are a kind of rejecting futher access from IP addresses for some period upon authentication failure. Therefore, if attacker uses distributed IP addresses, such methods could be run through. On the contrary, the method described in this page splits authentication into 2 stages. Therefore, the system won't be damaged even if the first stage was run through. Also, since the second stage has a lot of flexibility and has no need to disclose the authentication method, it is quite difficult to use brute force attacks.

Step 1: Compiling the program

Decide the environment variable's name and its value which act as password. In this page, we assume the environment variable name is "CERBERUS" and its value is "sftp".

Compile the below program. We assume the location of compiled program as /bin/env_chk . We assume the location of SSH server program as /usr/sbin/sshd .

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <syslog.h>
#include <string.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>

#define SECRET_ENVIRONMENT "CERBERUS=sftp"
#define SSHD_EXECUTABLE_PATH "/usr/sbin/sshd"

int main(int raw_argc, char *raw_argv[])
{
	int i;
	int argc;
	int envc;
	char *filename;
	char **argv;
	char **envp;
	{ /* Check that I'm an execute handler process.  */
		int fd = open("/proc/ccs/.execute_handler", O_RDONLY);
		close(fd);
		if (fd == EOF) {
			fprintf(stderr, "FATAL: I'm not execute_handler.\n");
			return 1;
		}
	}
	if (raw_argc < 7)
		return 1;
	filename = raw_argv[4];
	argc = atoi(raw_argv[5]);
	envc = atoi(raw_argv[6]);
	if (raw_argc != argc + envc + 7)
		return 1;
	for (i = 5; i < argc + 5; i++)
		raw_argv[i] = raw_argv[i + 2];
	raw_argv[argc + 5] = NULL;
	for (i = argc + 6; i < argc + envc + 6; i++)
		raw_argv[i] = raw_argv[i + 1];
	raw_argv[argc + envc + 6] = NULL;
	argv = raw_argv + 5;
	envp = raw_argv + argc + 6;
	/* "/usr/sbin/sshd" executes "/usr/sbin/sshd" with "-R" option. */
	if (argc == 2 && !strcmp(argv[1], "-R") &&
	    !strcmp(raw_argv[2], SSHD_EXECUTABLE_PATH) &&
	    !strcmp(filename, SSHD_EXECUTABLE_PATH)) {
		execve(filename, argv, envp);
		return 1;
	}
	/*
	 * Check environment variables passed to execve() request
	 * and execute the program if it has SECRET_ENVIRONMENT environment.
	 */
	for (i = 0; i < envc; i++) {
		if (strcmp(envp[i], SECRET_ENVIRONMENT))
			continue;
		while (i < envc) {
			envp[i] = envp[i + 1];
			i++;
		}
		execve(filename, argv, envp);
		break;
	}
	return 1;
}

Step 2: Making SSH to pass and receive environment variables

Append the name of environment variable which SSH server accepts to /etc/ssh/sshd_config .

AcceptEnv CERBERUS

Append the name of environment variable which SSH client sends to /etc/ssh/ssh_config .

SendEnv CERBERUS

Step 3: Install and initialize TOMOYO Linux

Install TOMOYO Linux and run below commandline in order to initialize TOMOYO Linux.

# /usr/lib/ccs/init_policy

Then, please do below operations before you reboot using TOMOYO Linux kernel.

Append below line to /etc/ccs/exception_policy.conf in order to initialize domain transition when /usr/sbin/sshd is executed.

initialize_domain /usr/sbin/sshd from any

Append below line to /etc/ccs/domain_policy.conf so that program execution requests from /usr/sbin/sshd are passed to /bin/env_chk .

<kernel> /usr/sbin/sshd

task auto_execute_handler /bin/env_chk

Step 4: Operation

Now, you are ready to start operation. Please reboot using TOMOYO Linux kernel.

Access the SSH server using sftp program, and verify that the connection is closed immediately after the SSH authentication.

Set environment variable "CERBERUS" with the value "sftp".

Access the SSH server using sftp program, and verify that you can operate.

Explanation

TOMOYO Linux's execute_handler functionality intercepts program execution requests from /usr/sbin/sshd and passes the program execution requests to /bin/env_chk . Then, /bin/env_chk validates parameters and execute the requested program only when appropriate parameters are provided.

This checking is applied after SSH authentication. Thus, operations without shell execution (e.g. TCP port forwarding) and operations after SSH login shell programs are permitted. If you want to restrict operations without shell execution or operations after SSH login shell, you need to develop policy for SSH server process and SSH login shells and apply enforcing mode.

Application idea

Since the movie above access SSH server for executing sftp service, it uses "sftp" as password. But there is no need to use fixed password. You may change behavior depending on the password. For example, you can provide download only sftp service if the environment variable "ssh_type" is set to "ro-sftp", uploadable sftp service if the environment variable "ssh_type" is set to "rw-sftp".

Also, you can audit parameters passed to program execution requests. For example, you can audit the name of requested programs and commandline arguments and environment variables of programs executed from SSH login shell.