Hardening SSH: Keys, Config and Fail2ban in Practice

SSH hardening is the work of turning a default OpenSSH server, which happily accepts password logins from the entire internet, into one that only your keys can reach and that bans the rest after a few tries. This guide walks the whole chain end to end on a lab box: a strong ed25519 key, password authentication switched off, the handful of sshd_config directives that actually move the needle, sane modern crypto, a fail2ban jail for the sshd service, optional multi-factor, and a safe way to apply it all without getting locked out. Every example uses the lab host lab.0xpriv.dev and the internal range 10.10.10.0/24.

Read it top to bottom the first time to understand why each setting exists, then keep it open as a checklist when you build the next server. Nothing here is exotic. It is the baseline every box you run should start from.

01Why SSH is a top target and what hardening achieves

SSH hardening matters because port 22 is one of the most probed services on the public internet, and a default configuration gives attackers a free guessing game. Any host with a routable address starts collecting login attempts within minutes of coming online. The bots are not clever, they are relentless: they work through lists of common usernames and leaked passwords, thousands of tries an hour, hoping one account was set up lazily. A single weak password on one account is all they need.

Hardening changes the shape of that problem. The goal is not to make SSH invisible, it is to make guessing pointless and noise cheap to absorb. When the server only accepts a cryptographic key, a password list is worthless no matter how long the bots run. When a rate limiter bans an address after a few failures, the volume of attempts drops to near zero. And when logging is turned up, the handful of attempts that do land leave a clear trail. Those three ideas, key-only access, automatic banning, and good logging, are the whole of this guide.

One caveat worth stating up front: hardening SSH reduces exposure, it does not replace the rest of your defenses. A compromised key, a backdoored package, or a vulnerable app on the same host still gets someone in. Treat what follows as closing the front door properly, not as the only lock in the building.

02Key-based authentication and a strong ed25519 key

Key-based authentication replaces a memorized secret with a key pair, and it is the single change that does the most for your security. You hold a private key, the server holds the matching public key, and a login succeeds only when the client proves it owns the private half. There is no shared secret traveling the wire and nothing short enough for a human to guess. The modern default is ed25519: short, fast, and based on a well-regarded elliptic curve.

Generate the pair on your own machine, never on the server. Add a passphrase so the private key is useless if the file is stolen, and let your agent cache it so you only type it once per session.

$ ssh-keygen -t ed25519 -a 100 -C "[email protected]"
Generating public/private ed25519 key pair.
Enter passphrase (empty for no passphrase): ********
Your identification has been saved in /home/you/.ssh/id_ed25519
Your public key has been saved in /home/you/.ssh/id_ed25519.pub
$ ssh-copy-id -i ~/.ssh/id_ed25519.pub [email protected]
Number of key(s) added: 1
Now try logging in: ssh '[email protected]'

The -a 100 flag sets the number of KDF rounds used to protect the private key on disk, so a stolen key file resists brute forcing of its passphrase. If you still run older hardware that cannot do ed25519, an RSA key of at least 3072 bits (ssh-keygen -t rsa -b 4096) is an acceptable fallback, but prefer ed25519 everywhere it is available.

Confirm the key login works before you change anything else. If ssh [email protected] drops you to a shell without asking for the account password, the server now trusts your key and you are clear to turn passwords off.

03Disabling password authentication safely

Disabling password authentication is the step that makes every stolen-password list irrelevant, but it is also the one most likely to lock you out if you rush it. The rule is simple: prove the key works first, in a session you keep open, and only then flip the setting. Never disable passwords from a connection you have not already re-established with a key.

On a current OpenSSH install the cleanest place to put overrides is a drop-in file under /etc/ssh/sshd_config.d/, which is included automatically and survives package upgrades of the main config. The two directives that matter here are the password switch and the keyboard-interactive switch, because leaving the second one on can quietly re-enable password prompts through PAM.

# /etc/ssh/sshd_config.d/10-no-passwords.conf   # 0xpriv.dev/defense
PubkeyAuthentication yes
PasswordAuthentication no
KbdInteractiveAuthentication no
PermitEmptyPasswords no

Apply it with a validate-then-reload cycle, which the next sections return to, and test a fresh login in a second terminal before you trust it. If key auth is set up correctly, a password attempt from a bot now gets refused before it ever reaches a credential check.

04The sshd_config settings that matter

The sshd_config settings that matter are a short list, and chasing every obscure directive is a waste of time next to getting these right. Each one below removes a specific piece of attack surface. Here is what they do and why they belong in your baseline.

Put them together into one block. AllowGroups is usually easier to manage than a per-user list: create a group, add the accounts that need SSH, and the policy stays in one place.

# /etc/ssh/sshd_config.d/20-hardening.conf   # 0xpriv.dev/defense
PermitRootLogin no
PasswordAuthentication no
KbdInteractiveAuthentication no
PubkeyAuthentication yes
PermitEmptyPasswords no

# who may connect at all
AllowGroups ssh-users
# or pin individual accounts, optionally to a source:
# AllowUsers [email protected]/24

# slow down and bound each attempt
MaxAuthTries 3
LoginGraceTime 20

# drop features a server does not need
X11Forwarding no
AllowTcpForwarding no

# notice idle sessions and log verbosely
ClientAliveInterval 300
ClientAliveCountMax 2
LogLevel VERBOSE

Two additions there earn their place. LogLevel VERBOSE records the fingerprint of the key used for each login, which is gold when you later need to answer "who actually connected". ClientAliveInterval and ClientAliveCountMax together close sessions that have gone silent, so a forgotten laptop does not hold an authenticated session open forever.

05Modern key exchange and cipher choices

Key exchange and cipher choices decide which algorithms the server will negotiate, and the aim is to offer only modern, well-reviewed ones. Current OpenSSH already defaults to a sensible set, so most people do not need to touch this. You pin an explicit list when a compliance baseline demands it, or when you want to drop older algorithms that are still enabled for compatibility. Set it and you refuse any client that only offers something weaker.

# /etc/ssh/sshd_config.d/30-crypto.conf   # 0xpriv.dev/defense
KexAlgorithms curve25519-sha256,[email protected]
Ciphers [email protected],[email protected],[email protected]
MACs [email protected],[email protected]
HostKeyAlgorithms ssh-ed25519,rsa-sha2-512

The pattern across all four lines is the same: prefer the curve25519 key exchange, authenticated-encryption ciphers such as ChaCha20-Poly1305 and AES-GCM, and encrypt-then-MAC (the -etm suffix) integrity checks. Rather than memorize a list that shifts as algorithms age out, copy the current recommended set from Mozilla's OpenSSH guidelines, which tracks what is safe and is linked at the end. Check what your build supports with ssh -Q kex, ssh -Q cipher, and ssh -Q mac before pinning, so you do not list something the binary cannot provide.

06Adding fail2ban for the sshd jail

A fail2ban jail watches the auth log and bans any address that fails to log in too many times, which turns a flood of guesses into a trickle. Even with passwords off, bots keep knocking, and each refused connection is still noise in your logs and load on the daemon. fail2ban reads the log, matches the failure pattern with its built-in sshd filter, and tells the firewall to drop the offending address for a while.

Install it, then put your settings in jail.local rather than editing jail.conf, because jail.local overrides the shipped defaults and is not overwritten on upgrade.

# /etc/fail2ban/jail.local   # 0xpriv.dev/defense
[DEFAULT]
# never ban yourself or the internal range
ignoreip = 127.0.0.1/8 ::1 10.10.10.0/24
findtime = 10m
maxretry = 4
bantime  = 1h
# make repeat offenders wait progressively longer
bantime.increment = true
bantime.factor    = 2
bantime.maxtime   = 1w

[sshd]
enabled = true
port    = ssh
backend = systemd

That jail bans an address for an hour after four failures inside ten minutes, and each later ban for the same address lasts twice as long, up to a week. backend = systemd reads failures straight from the journal on a systemd host; on a system that writes a plain file instead, drop that line and set logpath = %(sshd_log)s. Reload and confirm the jail is live.

$ sudo systemctl enable --now fail2ban
$ sudo fail2ban-client status sshd
Status for the jail: sshd
|- Filter
|  |- Currently failed: 2
|  |- Total failed:     341
|  `- File list:        /var/log/auth.log
`- Actions
   |- Currently banned: 3
   |- Total banned:     57
   `- Banned IP list:   10.10.10.5 ...

07Optional multi-factor authentication

Multi-factor authentication adds a second proof on top of your key, so that even a stolen private key is not enough on its own. The common setup pairs your key with a time-based one-time code from an app, enforced by SSH requiring both methods to succeed. It is optional because key-only access is already strong, but on a jump host or anything facing the public internet the extra factor is worth the small friction.

Install the Google Authenticator PAM module, run google-authenticator as each user to create their secret and scan the QR code, then tell SSH to demand both a key and the interactive code.

# add to your hardening drop-in to require key AND a TOTP code
KbdInteractiveAuthentication yes
AuthenticationMethods publickey,keyboard-interactive

Note the trade-off with the earlier settings: turning on MFA means switching KbdInteractiveAuthentication back to yes, because that is the channel the one-time code travels over. The AuthenticationMethods publickey,keyboard-interactive line is what keeps it safe: the comma means both must pass, so the interactive step only ever asks for the second factor, never for an account password. You also add auth required pam_google_authenticator.so to the SSH PAM stack so the code is actually checked.

08Verifying the config without locking yourself out

Verifying the config safely is the habit that separates a routine change from a late-night console recovery. SSH is the one service where a bad edit can cut off the very connection you are using to fix it, so the workflow is built around never trusting a change until a second, independent session has proven it works. Follow these steps every time.

$ sudo sshd -t
$ echo $?
0
$ sudo systemctl reload ssh
$ # now open a NEW terminal and test before closing this one
$ ssh -v [email protected]
debug1: Authentication succeeded (publickey).

The -v flag on the test login is worth keeping. It shows exactly which methods the client offered and which the server accepted, so if something is off you see whether the key was rejected, a method was missing, or the account was simply not allowed.

09Detecting brute force in the auth log

Detecting brute force in the auth log comes down to recognizing one pattern: many failed logins from one address in a short window. On Debian and Ubuntu that record lives in /var/log/auth.log; on Red Hat family systems it is /var/log/secure, and on a systemd host you can read it with journalctl -u ssh. The lines you care about say Failed password or Invalid user, and counting them per source address surfaces the offenders fast.

# top source addresses by failed SSH login, newest log
sudo grep -E "Failed password|Invalid user" /var/log/auth.log \
  | grep -oE "from [0-9.]+" \
  | sort | uniq -c | sort -rn | head

# the same read straight from the journal on a systemd box
sudo journalctl -u ssh --since "1 hour ago" | grep -c "Failed password"

$ sudo grep "Failed password" /var/log/auth.log | grep -oE "from [0-9.]+" | sort | uniq -c | sort -rn | head
   412 from 10.10.10.5
     9 from 10.10.10.23
     2 from 10.10.10.8

That one address with hundreds of failures is the signature of an automated attack, and it is exactly what fail2ban is acting on in the background. The reason to also watch the log yourself is that a clean log is itself a signal: once hardening is in place and passwords are off, those Failed password lines should nearly vanish, because the server rejects password attempts before they become a credential check. A sudden return of that noise is worth a look.

10The SSH hardening checklist

The hardening checklist below is the whole guide boiled down to actions you can run through on any new server. Work top to bottom, verify with a second session at each risky step, and you end up with a box that ignores the guessing game entirely.

11Frequently asked questions

12Wrapping up

Hardening SSH is not a long project, it is a short checklist you run once per server and then forget. Put a passphrase-protected ed25519 key on the box, turn off passwords, deny root, limit who may connect, add a fail2ban jail, turn logging up, and verify every change from a second session before you trust it. Do that and the endless guessing traffic on port 22 becomes background noise your server simply drops, while your own access stays as simple as one command. Build the next machine from the same list and good SSH posture becomes a habit rather than a scramble.