Category: Cron EN

A Permanent IP Blacklist with Fail2ban, UFW, and Ipset

Introduction: Beyond Temporary Protection

In the digital world, where server attacks are a daily occurrence, merely reacting is not enough. Although tools like Fail2ban provide a basic line of defence, their temporary blocks leave a loophole—persistent attackers can return and try again after the ban expires. This article provides a detailed guide to building a fully automated, two-layer system that turns ephemeral bans into permanent, global blocks. The combination of Fail2ban, UFW, and the powerful Ipset tool creates a mechanism that permanently protects your server from known repeat offenders.

Layer One: Reaction with Fail2ban

At the start of every attack is Fail2ban. This daemon monitors log files (e.g., sshd.log, apache.log) for patterns indicating break-in attempts, such as multiple failed login attempts. When it detects such activity, it immediately blocks the attacker’s IP address by adding it to the firewall rules for a defined period (e.g., 10 minutes, 30 days). This is an effective but short-term response.

Layer Two: Persistence with UFW and Ipset

For a ban to become permanent, we need a more robust, centralised method of managing IP addresses. This is where UFW and Ipset come in.

What is Ipset?

Ipset is a Linux kernel extension that allows you to manage sets of IP addresses, networks, or ports. It is a much more efficient solution than adding thousands of individual rules to a firewall. Instead, the firewall can refer to an entire set with a single rule.

Ipset Installation and Configuration

The first step is to install Ipset on your system. We use standard package managers for this.

sudo apt update
sudo apt install ipset

Next, we create two sets: blacklist for IPv4 addresses and blacklist_v6 for IPv6.

sudo ipset create blacklist hash:ip hashsize 4096
sudo ipset create blacklist_v6 hash:net family inet6 hashsize 4096

The hashsize parameter determines the maximum number of entries, which is crucial for performance.

Integrating Ipset with the UFW Firewall

For UFW to start using our sets, we must add the appropriate commands to its rules. We edit the UFW configuration files, adding rules that block traffic originating from addresses contained in our Ipset sets. For IPv4, we edit /etc/ufw/before.rules:

sudo nano /etc/ufw/before.rules

Immediately after *filter and :ufw-before-input [0:0], add:

# Rules for the permanent blacklist (ipset)
# Block any incoming traffic from IP addresses in the ‘blacklist’ set (IPv4)
-A ufw-before-input -m set –match-set blacklist src -j DROP

For IPv6, we edit /etc/ufw/before6.rules:

sudo nano /etc/ufw/before6.rules

Immediately after *filter and :ufw6-before-input [0:0], add:

# Rules for the permanent blacklist (ipset) IPv6
# Block any incoming traffic from IP addresses in the ‘blacklist_v6’ set
-A ufw6-before-input -m set –match-set blacklist_v6 src -j DROP

After adding the rules, we reload UFW for them to take effect:

sudo ufw reload

Script for Automatic Blacklist Updates

The core of the system is a script that acts as a bridge between Fail2ban and Ipset. Its job is to collect banned addresses, ensure they are unique, and synchronise them with the Ipset sets.

Create the script file:

sudo nano /usr/local/bin/update-blacklist.sh

Below is the content of the script. It works in several steps:

1. Creates a temporary, unique list of IP addresses from Fail2ban logs and the existing blacklist.
2. Creates temporary Ipset sets.
3. Reads addresses from the unique list and adds them to the appropriate temporary sets (distinguishing between IPv4 and IPv6).
4. Atomically swaps the old Ipset sets with the new, temporary ones, minimising the risk of protection gaps.
5. Destroys the old, temporary sets.
6. Returns a summary of the number of blocked addresses.

#!/bin/bash

BLACKLIST_FILE=”/etc/fail2ban/blacklist.local”
IPSET_NAME_V4=”blacklist”
IPSET_NAME_V6=”blacklist_v6″

touch “$BLACKLIST_FILE”

# Create a unique list of banned IPs from the log and the existing blacklist file
(grep ‘Ban’ /var/log/fail2ban.log | awk ‘{print $(NF)}’ && cat “$BLACKLIST_FILE”) | sort -u > “$BLACKLIST_FILE.tmp”
mv “$BLACKLIST_FILE.tmp” “$BLACKLIST_FILE”

# Create temporary ipsets
sudo ipset create “${IPSET_NAME_V4}_tmp” hash:ip hashsize 4096 –exist
sudo ipset create “${IPSET_NAME_V6}_tmp” hash:net family inet6 hashsize 4096 –exist

# Add IPs to the temporary sets
while IFS= read -r ip; do
    if [[ “$ip” == *”:”* ]]; then
        sudo ipset add “${IPSET_NAME_V6}_tmp” “$ip”
    else
        sudo ipset add “${IPSET_NAME_V4}_tmp” “$ip”
    fi
done < “$BLACKLIST_FILE”

# Atomically swap the temporary sets with the active ones
sudo ipset swap “${IPSET_NAME_V4}_tmp” “$IPSET_NAME_V4”
sudo ipset swap “${IPSET_NAME_V6}_tmp” “$IPSET_NAME_V6”

# Destroy the temporary sets
sudo ipset destroy “${IPSET_NAME_V4}_tmp”
sudo ipset destroy “${IPSET_NAME_V6}_tmp”

# Count the number of entries
COUNT_V4=$(sudo ipset list “$IPSET_NAME_V4” | wc -l)
COUNT_V6=$(sudo ipset list “$IPSET_NAME_V6” | wc -l)

# Subtract header lines from count
let COUNT_V4=$COUNT_V4-7
let COUNT_V6=$COUNT_V6-7

# Ensure count is not negative
[ $COUNT_V4 -lt 0 ] && COUNT_V4=0
[ $COUNT_V6 -lt 0 ] && COUNT_V6=0

echo “Blacklist and ipset updated. Blocked IPv4: $COUNT_V4, Blocked IPv6: $COUNT_V6”
exit 0

After creating the script, give it execute permissions:

sudo chmod +x /usr/local/bin/update-blacklist.sh

Automation and Persistence After a Reboot

To run the script without intervention, we use a cron schedule. Open the crontab editor for the root user and add a rule to run the script every hour:

sudo crontab -e

Add this line:

0 * * * * /usr/local/bin/update-blacklist.sh

Or to run it once a day at 6 a.m.:

0 6 * * * /usr/local/bin/update-blacklist.sh

The final, crucial step is to ensure the Ipset sets survive a reboot, as they are stored in RAM by default. We create a systemd service that will save their state before the server shuts down and load it again on startup.

sudo nano /etc/systemd/system/ipset-persistent.service
“`ini
[Unit]
Description=Saves and restores ipset sets on boot/shutdown
Before=network-pre.target
ConditionFileNotEmpty=/etc/ipset.rules

[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/bin/bash -c “/sbin/ipset create blacklist hash:ip –exist; /sbin/ipset create blacklist_v6 hash:net family inet6 –exist; /sbin/ipset restore -f /etc/ipset.rules”
ExecStop=/sbin/ipset save -f /etc/ipset.rules

[Install]
WantedBy=multi-user.target

Finally, enable and start the service:

sudo systemctl daemon-reload
sudo systemctl enable –now ipset-persistent.service

How Does It Work in Practice?

The entire system is an automated chain of events that works in the background to protect your server from attacks. Here is the flow of information and actions:

1. Attack Response (Fail2ban):

• Someone tries to break into the server (e.g., by repeatedly entering the wrong password via SSH).
• Fail2ban, monitoring system logs (/var/log/fail2ban.log), detects this pattern.
• It immediately adds the attacker’s IP address to a temporary firewall rule, blocking their access for a specified time.

2. Permanent Banning (Script and Cron):

• Every hour (as set in cron), the system runs the update-blacklist.sh script.
• The script reads the Fail2ban logs, finds all addresses that have been banned (lines containing “Ban”), and then compares them with the existing local blacklist (/etc/fail2ban/blacklist.local).
• It creates a unique list of all banned addresses.
• It then creates temporary ipset sets (blacklist_tmp and blacklist_v6_tmp) and adds all addresses from the unique list to them.
• It performs an ipset swap operation, which atomically replaces the old, active sets with the new, updated ones.
• UFW, thanks to the previously defined rules, immediately starts blocking the new addresses that have appeared in the updated ipset sets.

3. Persistence After Reboot (systemd Service):

• Ipset’s operation is volatile—the sets only exist in memory. The ipset-persistent.service solves this problem.
• Before shutdown/reboot: systemd runs the ExecStop=/sbin/ipset save -f /etc/ipset.rules command. This saves the current state of all ipset sets to a file on the disk.
• After power-on/reboot: systemd runs the ExecStart command, which restores the sets. It reads all blocked addresses from the /etc/ipset.rules file and automatically recreates the ipset sets in memory.

Thanks to this, even if the server is rebooted, the IP blacklist remains intact, and protection is active from the first moments after the system starts.

Summary and Verification

The system you have built is a fully automated, multi-layered protection mechanism. Attackers are temporarily banned by Fail2ban, and their addresses are automatically added to a permanent blacklist, which is instantly blocked by UFW and Ipset. The systemd service ensures that the blacklist survives server reboots, protecting against repeat offenders permanently. To verify its operation, you can use the following commands:

sudo ufw status verbose
sudo ipset list blacklist
sudo ipset list blacklist_v6
sudo systemctl status ipset-persistent.service

How to Create a Reliable IP Whitelist in UFW and Ipset

Introduction: Why a Whitelist is Crucial

When configuring advanced firewall rules, especially those that automatically block IP addresses (like in systems with Fail2ban), there is a risk of accidentally blocking yourself or key services. A whitelist is a mechanism that acts like a VIP pass for your firewall—IP addresses on this list will always have access, regardless of other, more restrictive blocking rules.

This guide will show you, step-by-step, how to create a robust and persistent whitelist using UFW (Uncomplicated Firewall) and ipset. As an example, we will use the IP address 111.222.333.444, which we want to add as trusted.

Step 1: Create a Dedicated Ipset Set for the Whitelist

The first step is to create a separate “container” for our trusted IP addresses. Using ipset is much more efficient than adding many individual rules to iptables.

Open a terminal and enter the following command:

sudo ipset create whitelist hash:ip

What did we do?

• ipset create: The command to create a new set.
• whitelist: The name of our set. It’s short and unambiguous.
• hash:ip: The type of set. hash:ip is optimised for storing and very quickly looking up single IPv4 addresses.

Step 2: Add a Trusted IP Address

Now that we have the container ready, let’s add our example trusted IP address to it.

sudo ipset add whitelist 111.222.333.444

You can repeat this command for every address you want to add to the whitelist. To check the contents of the list, use the command:

sudo ipset list whitelist

Step 3: Modify the Firewall – Giving Priority to the Whitelist

This is the most important step. We need to modify the UFW rules so that connections from addresses on the whitelist are accepted immediately, before the firewall starts processing any blocking rules (including those from the ipset blacklist or Fail2ban).

Open the before.rules configuration file. This is the file where rules processed before the main UFW rules are located.

sudo nano /etc/ufw/before.rules

Go to the beginning of the file and find the *filter section. Just below the :ufw-before-input [0:0] line, add our new snippet. Placing it at the very top ensures it will be processed first.

*filter
:ufw-before-input [0:0]
# Rule for the whitelist (ipset) ALWAYS HAS PRIORITY
# Accept any traffic from IP addresses in the ‘whitelist’ set
-A ufw-before-input -m set –match-set whitelist src -j ACCEPT

• -A ufw-before-input: We add the rule to the ufw-before-input chain.
• -m set –match-set whitelist src: Condition: if the source (src) IP address matches the whitelist set…
• -j ACCEPT: Action: “immediately accept (ACCEPT) the packet and stop processing further rules for this packet.”

Save the file and reload UFW:

sudo ufw reload

From this point on, any connection from the address 111.222.333.444 will be accepted immediately.

Step 4: Ensuring Whitelist Persistence

Ipset sets are stored in memory and disappear after a server reboot. To make our whitelist persistent, we need to ensure it is automatically loaded every time the system starts. We will use our previously created ipset-persistent.service for this.

Update the systemd service to “teach” it about the existence of the new whitelist set.

sudo nano /etc/systemd/system/ipset-persistent.service

Find the ExecStart line and add the create command for whitelist. If you already have other sets, simply add whitelist to the line. An example of an updated line:

ExecStart=/bin/bash -c “/sbin/ipset create whitelist hash:ip –exist; /sbin/ipset create blacklist hash:ip –exist; /sbin/ipset create blacklist_v6 hash:net family inet6 –exist; /sbin/ipset restore -f /etc/ipset.rules”

Reload the systemd configuration:

sudo systemctl daemon-reload

Save the current state of all sets to the file. This command will overwrite the old /etc/ipset.rules file with a new version that includes information about your whitelist.

sudo ipset save > /etc/ipset.rules

Restart the service to ensure it is running with the new configuration:

sudo systemctl restart ipset-persistent.service

Summary

Congratulations! You have created a solid and reliable whitelist mechanism. With it, you can securely manage your server, confident that trusted IP addresses like 111.222.333.444 will never be accidentally blocked. Remember to only add fully trusted addresses to this list, such as your home or office IP address.

How to Effectively Block IP Addresses and Subnets on a Linux Server

Blocking single IP addresses is easy, but what if attackers use multiple addresses from the same network? Manually banning each one is inefficient and time-consuming.

In this article, you will learn how to use ipset and iptables to effectively block entire subnets, automating the process and saving valuable time.

Why is Blocking Entire Subnets Better?

Many attacks, especially brute-force types, are carried out from multiple IP addresses belonging to the same operator or from the same pool of addresses (subnet). Blocking just one of them is like patching a small hole in a large dam—the rest of the traffic can still get through.

Instead, you can block an entire subnet, for example, 45.148.10.0/24. This notation means you are blocking 256 addresses at once, which is much more effective.

Script for Automatic Subnet Blocking

To automate the process, you can use the following bash script. This script is interactive—it asks you to provide the subnet to block, then adds it to an ipset list and saves it to a file, making the block persistent.

Let’s analyse the script step-by-step:

#!/bin/bash

# The name of the ipset list to which subnets will be added
BLACKLIST_NAME=”blacklist_nets”
# The file where blocked subnets will be appended
BLACKLIST_FILE=”/etc/fail2ban/blacklist_net.local”

# 1. Create the blacklist file if it doesn’t exist
touch “$BLACKLIST_FILE”

# 2. Check if the ipset list already exists. If not, create it.
# Using “hash:net” allows for storing subnets, which is key.
if ! sudo ipset list $BLACKLIST_NAME >/dev/null 2>&1; then
    sudo ipset create $BLACKLIST_NAME hash:net maxelem 65536
fi

# 3. Loop to prompt the user for subnets to block.
# The loop ends when the user types “exit”.
while true; do
    read -p “Enter the subnet address to block (e.g., 192.168.1.0/24) or type ‘exit’: ” subnet
    if [ “$subnet” == “exit” ]; then
        break
    elif [[ “$subnet” =~ ^([0-9]{1,3}\.){3}[0-9]{1,3}\/[0-9]{1,2}$ ]]; then
        # Check if the subnet is not already in the file to avoid duplicates
        if ! grep -q “^$subnet$” “$BLACKLIST_FILE”; then
            echo “$subnet” | sudo tee -a “$BLACKLIST_FILE” > /dev/null
            # Add the subnet to the ipset list
            sudo ipset add $BLACKLIST_NAME $subnet
            echo “Subnet $subnet added.”
        else
            echo “Subnet $subnet is already on the list.”
        fi
    else
        # Check if the entered format is correct
        echo “Error: Invalid format. Please provide the address in ‘X.X.X.X/Y’ format.”
    fi
done

# 4. Add a rule in iptables that blocks all traffic from addresses on the ipset list.
# This ensures the rule is added only once.
if ! sudo iptables -C INPUT -m set –match-set $BLACKLIST_NAME src -j DROP >/dev/null 2>&1; then
    sudo iptables -I INPUT -m set –match-set $BLACKLIST_NAME src -j DROP
fi

# 5. Save the iptables rules to survive a reboot.
# This part checks which tool the system uses.
if command -v netfilter-persistent &> /dev/null; then
    sudo netfilter-persistent save
elif command -v service &> /dev/null && service iptables status >/dev/null 2>&1; then
    sudo service iptables save
fi

echo “Script finished. The ‘$BLACKLIST_NAME’ list has been updated, and the iptables rules are active.”

How to Use the Script

1. Save the script: Save the code above into a file, e.g., block_nets.sh.
2. Give permissions: Make sure the file has execute permissions: chmod +x block_nets.sh.
3. Run the script: Execute the script with root privileges: sudo ./block_nets.sh.
4. Provide subnets: The script will prompt you to enter subnet addresses. Simply type them in the X.X.X.X/Y format and press Enter. When you are finished, type exit.

Ensuring Persistence After a Server Reboot

Ipset sets are stored in RAM by default and disappear after a server restart. For the blocked addresses to remain active, you must use a systemd service that will load them at system startup.

If you already have such a service (e.g., ipset-persistent.service), you must update it to include the new blacklist_nets list.

1. Edit the service file: Open your service’s configuration file.
   sudo nano /etc/systemd/system/ipset-persistent.service
   
2. Update the ExecStart line: Find the ExecStart line and add the create command for the blacklist_nets set. An example updated ExecStart line should look like this (including previous sets):
   ExecStart=/bin/bash -c “/sbin/ipset create whitelist hash:ip –exist; /sbin/ipset create blacklist hash:ip –exist; /sbin/ipset create blacklist_v6 hash:net family inet6 –exist; /sbin/ipset create blacklist_nets hash:net –exist; /sbin/ipset restore -f /etc/ipset.rules”
   
3. Reload the systemd configuration:
   sudo systemctl daemon-reload
   
4. Save the current state of all sets to the file: This command will overwrite the old /etc/ipset.rules file with a new version that contains information about all your lists, including blacklist_nets.
   sudo ipset save > /etc/ipset.rules
   
5. Restart the service:
   sudo systemctl restart ipset-persistent.service
   

With this method, you can simply and efficiently manage your server’s security, effectively blocking entire subnets that show suspicious activity, and be sure that these rules will remain active after every reboot.