Every day and night, the SSH login by key into my public accessible Oracle Cloud Infrastructure Linux Compute Instance was permitted for hours. And sometimes, when I had luck, it worked. For me it was not clear when it works and when not. But something has blocked me. The password authentification in the OCI Linux instance is basically disabled, the key is the only way to log in.
After some investigation on the OCI instance, I found a huge amount of login trials in the /var/log/secure file. These brute force attacks were locking me out!
Nov 4 03:57:24 instance-as-1 sshd[1975]: Received disconnect from 132.232.17.146 port 53924:11: Normal Shutdown, Thank you for playing [preauth] Nov 4 03:57:24 instance-as-1 sshd[1975]: Disconnected from 132.232.17.146 port 53924 [preauth] Nov 4 04:01:22 instance-as-1 sshd[2194]: Received disconnect from 132.232.17.146 port 59916:11: Normal Shutdown, Thank you for playing [preauth] Nov 4 04:01:22 instance-as-1 sshd[2194]: Disconnected from 132.232.17.146 port 59916 [preauth] Nov 4 04:05:28 instance-as-1 sshd[2448]: Received disconnect from 132.232.17.146 port 37640:11: Normal Shutdown, Thank you for playing [preauth] Nov 4 04:05:28 instance-as-1 sshd[2448]: Disconnected from 132.232.17.146 port 37640 [preauth] Nov 4 04:20:28 instance-as-1 sshd[3206]: Received disconnect from 125.65.42.187 port 48506:11: [preauth] Nov 4 04:20:28 instance-as-1 sshd[3206]: Disconnected from 125.65.42.187 port 48506 [preauth] Nov 4 04:21:47 instance-as-1 sshd[3275]: Did not receive identification string from 46.101.174.170 port 46073 Nov 4 04:21:47 instance-as-1 sshd[3272]: Received disconnect from 118.123.15.142 port 38448:11: [preauth] Nov 4 04:21:47 instance-as-1 sshd[3272]: Disconnected from 118.123.15.142 port 38448 [preauth] Nov 4 04:24:33 instance-as-1 sshd[3409]: Received disconnect from 118.123.15.210 port 41616:11: [preauth] Nov 4 04:24:33 instance-as-1 sshd[3409]: Disconnected from 118.123.15.210 port 41616 [preauth] Nov 4 04:24:48 instance-as-1 sshd[3421]: Received disconnect from 115.238.245.2 port 50782:11: [preauth] Nov 4 04:24:48 instance-as-1 sshd[3421]: Disconnected from 115.238.245.2 port 50782 [preauth] Nov 4 04:27:08 instance-as-1 sshd[3538]: Received disconnect from 61.184.247.2 port 59804:11: [preauth] Nov 4 04:27:08 instance-as-1 sshd[3538]: Disconnected from 61.184.247.2 port 59804 [preauth] Nov 4 04:32:07 instance-as-1 sshd[3785]: Received disconnect from 125.65.42.192 port 41143:11: [preauth] Nov 4 04:32:07 instance-as-1 sshd[3785]: Disconnected from 125.65.42.192 port 41143 [preauth] Nov 4 04:32:35 instance-as-1 sshd[3811]: Received disconnect from 61.184.247.11 port 57407:11: [preauth] Nov 4 04:32:35 instance-as-1 sshd[3811]: Disconnected from 61.184.247.11 port 57407 [preauth] Nov 4 04:36:02 instance-as-1 sshd[3981]: Received disconnect from 200.46.254.107 port 57063:11: Normal Shutdown, Thank you for playing [preauth] Nov 4 04:36:02 instance-as-1 sshd[3981]: Disconnected from 200.46.254.107 port 57063 [preauth] Nov 4 04:36:08 instance-as-1 sshd[3965]: Connection closed by 125.65.42.178 port 60537 [preauth] Nov 4 04:36:49 instance-as-1 sshd[4020]: Received disconnect from 115.238.245.8 port 51409:11: [preauth] Nov 4 04:36:49 instance-as-1 sshd[4020]: Disconnected from 115.238.245.8 port 51409 [preauth] Nov 4 04:39:32 instance-as-1 sshd[4168]: Invalid user pos from 200.46.254.107 port 50308 Nov 4 04:39:32 instance-as-1 sshd[4168]: input_userauth_request: invalid user pos [preauth] Nov 4 04:39:32 instance-as-1 sshd[4168]: Received disconnect from 200.46.254.107 port 50308:11: Normal Shutdown, Thank you for playing [preauth] Nov 4 04:39:32 instance-as-1 sshd[4168]: Disconnected from 200.46.254.107 port 50308 [preauth] Nov 4 04:39:46 instance-as-1 sshd[4179]: Received disconnect from 122.226.181.165 port 57930:11: [preauth] Nov 4 04:39:46 instance-as-1 sshd[4179]: Disconnected from 122.226.181.165 port 57930 [preauth] Nov 4 04:41:00 instance-as-1 sshd[4250]: Received disconnect from 115.238.245.14 port 42326:11: [preauth] Nov 4 04:41:00 instance-as-1 sshd[4250]: Disconnected from 115.238.245.14 port 42326 [preauth] Nov 4 04:41:17 instance-as-1 sshd[4265]: Received disconnect from 61.184.247.5 port 51434:11: [preauth] Nov 4 04:41:17 instance-as-1 sshd[4265]: Disconnected from 61.184.247.5 port 51434 [preauth]
There is a interesting OCI documentation available called Securing Compute with steps how to secure OCI compute cloud instances – and one of this recommendation is: install fail2ban.
https://docs.cloud.oracle.com/iaas/Content/Security/Reference/compute_security.htm
fail2ban
fail2ban is an open source tool which reads several types of logfiles and creates based on rules new entries in the firewall table to block remote connections. I has default filters for ssh, apache postfix and many more. From Wikipedia:
Fail2Ban is an intrusion prevention software framework that protects computer servers from brute-force attacks.
Link: https://www.fail2ban.org/wiki/index.php/Main_Page
Installation
All steps have to be executed as user root. FYI: I wanted to be informed when a new IP was banned, therefore I have installed sendmail too.
# yum install https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm # yum install fail2ban # yum install sendmail # systemctl enable fail2ban # systemctl start sendmail # systemctl enable sendmail
Configuration
For my fail2ban configuration I have created a new file called jail.local and made my settings there.
# cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local
jail.local
After 2 unsuccesful logins, the source IP will be banned for 86400 seconds. And if a new IP is added to the ban list, I get an email.
[DEFAULT] sender = OCI destemail = root action = %(action_mwl)s maxretry = 2 destemail = <my_email_address> bantime = 86400 [sshd] enabled = true
/etc/fail2ban/jail.d/00-firewalld.conf
For OL7 where firewalld is used, verify if the command firewallcmd-ipset is set in /etc/fail2ban/jail.d/00-firewalld.conf. If you use iptables, the command can be changed. Please read the documentation how to change the firewall.
[DEFAULT] banaction = firewallcmd-ipset
Start fail2ban
# systemctl start fail2ban
Verification
Status Check
# fail2ban-client status Status |- Number of jail: 1 `- Jail list: sshd
Status Check with details, there is already one IP listed.
[root@instance-as-1 ]# fail2ban-client status sshd Status for the jail: sshd |- Filter | |- Currently failed: 0 | |- Total failed: 7 | `- Journal matches: _SYSTEMD_UNIT=sshd.service + _COMM=sshd `- Actions |- Currently banned: 1 |- Total banned: 1 `- Banned IP list: 46.101.204.153
After some minutes, more entries were recognized in the /var/log/secure log file and added.
# tail -f /var/log/fail2ban.log 2018-11-05 07:54:10,467 fail2ban.filter [13219]: INFO [sshd] Found 46.101.204.153 2018-11-05 07:54:10,472 fail2ban.filter [13219]: INFO [sshd] Found 46.101.204.153 2018-11-05 07:54:10,478 fail2ban.filter [13219]: INFO [sshd] Found 46.101.204.153 2018-11-05 07:54:10,483 fail2ban.filter [13219]: INFO [sshd] Found 46.101.204.153 2018-11-05 07:54:11,050 fail2ban.actions [13219]: NOTICE [sshd] Ban 46.101.204.153
firewall-cmd
A new rule is automatically added with the match set failban-sshd.
# firewall-cmd --direct --get-all-rule ipv4 filter INPUT 0 -p tcp -m multiport --dports ssh -m set --match-set fail2ban-sshd src -j REJECT --reject-with icmp-port-unreachable
sshd Configuration
After the fail2ban installation, there were other entries left in /var/log/secure.
Nov 6 04:50:25 instance-as-1 sshd[5885]: Received disconnect from 209.141.51.85 port 38056:11: Bye Bye [preauth] Nov 6 04:50:25 instance-as-1 sshd[5885]: Disconnected from 209.141.51.85 port 38056 [preauth] Nov 6 04:50:27 instance-as-1 sshd[5887]: reverse mapping checking getaddrinfo for offshore.onion [209.141.51.85] failed - POSSIBLE BREAK-IN ATTEMPT! Nov 6 04:50:27 instance-as-1 sshd[5887]: Received disconnect from 209.141.51.85 port 39492:11: Bye Bye [preauth] Nov 6 04:50:27 instance-as-1 sshd[5887]: Disconnected from 209.141.51.85 port 39492 [preauth] Nov 6 04:50:28 instance-as-1 sshd[5889]: reverse mapping checking getaddrinfo for offshore.onion [209.141.51.85] failed - POSSIBLE BREAK-IN ATTEMPT! Nov 6 04:50:28 instance-as-1 sshd[5889]: Received disconnect from 209.141.51.85 port 40808:11: Bye Bye [preauth] Nov 6 04:50:28 instance-as-1 sshd[5889]: Disconnected from 209.141.51.85 port 40808 [preauth] Nov 6 04:50:30 instance-as-1 sshd[5891]: reverse mapping checking getaddrinfo for offshore.onion [209.141.51.85] failed - POSSIBLE BREAK-IN ATTEMPT! Nov 6 04:50:30 instance-as-1 sshd[5891]: Received disconnect from 209.141.51.85 port 42144:11: Bye Bye [preauth] Nov 6 04:50:30 instance-as-1 sshd[5891]: Disconnected from 209.141.51.85 port 42144 [preauth] Nov 6 04:50:31 instance-as-1 sshd[5893]: reverse mapping checking getaddrinfo for offshore.onion [209.141.51.85] failed - POSSIBLE BREAK-IN ATTEMPT!
After changing the parameter UseDNS to no in /etc/ssdh/sshd_config and a restart, these entries were history.
# vi /etc/ssh/sshd_config Set UseDNS no
# service sshd stop # service sshd start
Summary
Never let a OCI compute cloud running with a public IP without to monitor login attemps! fail2ban is one step to get more security. It is easy to configure and it helps a lot. But you have to do the basic work like software updates, SSH key enabling etc. The Oracle documentation is a good base to start! My next step will be to install and configure WAZUH – I keep you up to date!