```json
{
    "title": "Enterprise Shield – The Evolution – Part 1: What & Why",
    "url": "https://performancezen.com/2026/06/04/enterprise-shield-the-evolution-part-1/",
    "datePublished": "2026-06-04",
    "dateModified": "2026-06-05",
    "language": "en-US",
    "description": "Over the years, I have tried to limit the amount of spurious or aggressive traffic against my small, yet mighty web server environment. It started with a list of UFW…",
    "author": "spierzchala",
    "publisher": "Performance Zen"
}
```

# Enterprise Shield – The Evolution – Part 1: What & Why

Over the years, I have tried to limit the amount of spurious or aggressive traffic against my small, yet mighty web server environment. It started with a list of UFW rules, but that became very unwieldy once the list hit over 25,000 unique CIDR blocks.

A few months ago, working alongside some AI partners, I asked how this system could be made more efficient and manage even larger rulesets without making the server scream in pain every time a large burst of traffic hit it.

That's when the **Enterprise Shield** approach was born.

## What Is Enterprise Shield?

Enterprise Shield is a custom Linux firewall management system protecting `creaky2`, a vintage 2008 MacBook Pro repurposed as a personal Ubuntu Server hosting [performancezen.com](https://performancezen.com). It is not a product — it is a purpose-built system, grown organically over years of dealing with real-world attacks against a small but publicly visible WordPress blog.

At its core, Enterprise Shield does three things:

1. **Blocks hostile IP space** — autonomous systems (ASNs), country CIDR ranges, and individually flagged IPs — before they ever reach the web server.
2. **Rate-limits legitimate but noisy infrastructure** — Microsoft Azure, AWS, Google Cloud, Oracle Cloud, Cloudflare — so they can't hammer the server but aren't outright blocked.
3. **Logs, enriches, and reports** on everything that hits the firewall so you can understand the threat landscape over time.

The system wraps Linux's `ipset` and `iptables` primitives with custom tooling to make them manageable: automating blocklist maintenance, surviving reboots, and providing a real-time terminal dashboard and HTML threat analysis reports.

## A Brief History: How We Got Here

### The Early Days: Hand-Crafted BASH (v11.x)

Enterprise Shield started as a BASH script. Over time it grew significantly:

- **ASN blocking** via RADB WHOIS lookups, parallelised across threads
- **Country blocking** via the [ipverse/country-ip-blocks](https://github.com/ipverse/country-ip-blocks)

GitHub feed (43 countries)

- **AbuseIPDB integration** — a penalty box of individually-flagged malicious IPs, refreshed 5× daily (constrained by the free API tier)
- **Azure and hyperscaler separation** — Azure (`AS8075`) given its own rate-limit chain distinct from AWS/GCP/Oracle/Cloudflare
- **Atomic ipset swaps** — the live blocklist is never empty during an update
- **Delta safety abort** — if the new CIDR count drops >5% from the previous run, the script aborts rather than switching to a degraded ruleset
- **Systemd boot persistence** — two services that restore ipsets and iptables chains after every reboot

By version 11.25-MT, the script was around 500 lines, handled parallel WHOIS with a 4-thread batch model, and managed a blocklist of over **500,000 CIDR entries**.

It worked. But it had accumulated significant technical debt.

## Why We Rewrote It

### Problem 1: Every Run Was a Full Rebuild

The BASH system had no memory between runs. Every night at 2AM:

- **All ASN WHOIS data was re-fetched** — 447 ASNs × a RADB query each, regardless of whether anything had changed
- **All 43 country CIDR files were re-downloaded** from GitHub
- **All 500,000+ CIDR entries were re-inserted** from scratch into a staging ipset

This wasn't just slow — it was wasteful. ASN CIDR allocations change rarely. The ipverse country files change weekly at most. But the system had no way to say "I already know what AS3209 owns — skip it."

```
BASH nightly run cost (approximate): 447 ASN WHOIS lookups ~35 minutes 43 GitHub downloads ~2 minutes ipset full rebuild ~1 minute Total rebuild time: ~38 minutes Total per week: ~4.5 hours of server time
```

### Problem 2: State Was Scattered Across Flat Files

Runtime state was stored in flat files:

- `/etc/ipset.conf` — the full ipset dump (502,575 lines)
- `/etc/shield-iptables.rules` — the iptables chain rules
- `/var/lib/shield/last_entry_count` — just a number, for the delta check
- `/var/lib/shield/last_public_ip` — the server's external IP at last run

There was no audit trail, no history, no way to answer "what changed between last Tuesday and today?" The `last_entry_count` file was a single integer — useful for
the delta abort check, useless for anything else.

### Problem 3: No Hit Data

The BASH system blocked things — but didn't know what it blocked. There was no structured record of which IPs hit the firewall, when, or how many times. The `iptables -L` packet counters reset on every rebuild. A threat analysis report
was impossible.

### Problem 4: AbuseIPDB Was Bolted On

The AbuseIPDB penalty box was a separate script (`abuseipdb_penaltybox.sh`) that maintained its own ipset (`SHIELD_PENALTY`), with its own boot persistence service,
its own log file, and completely separate state from the main system. It worked, but it was an island.

### Problem 5: Boot Persistence Was Fragile

The two-service boot sequence (ipsets before UFW, chains after UFW) worked correctly but relied on `/etc/ipset.conf` — a 502,575-line flat file. If that file was corrupt or missing, the server came up with no firewall. There was no crash recovery, no verification that the restored state was sane.

## The Solution: Python + SQLite

The rewrite was designed around a single guiding principle:

> **The database is the system. Everything else derives from it.**

### What Changed

```
┌────────────────────────────────────────────────────────────-─────────┐ │ BASH v11.25-MT → Python v12 │ ├──────────────────────────┬────────────────────────────────-──────────┤ │ State storage │ Flat files → SQLite database │ │ Rebuild strategy │ Full re-fetch → Delta (only changed data) │ │ WHOIS cadence │ Every night for all ASNs → Only stale │ │ Country blocks │ Re-download always → ETag conditional GET │ │ AbuseIPDB │ Separate script/ipset → Integrated table │ │ Hit logging │ None → rsyslog → hits_parser → SQLite │ │ Boot persistence │ Flat file restore → DB-driven restore.py │ │ Crash recovery │ None → rebuild_in_progress flag in DB │ │ Config change detection │ None → SHA-256 hash of config files │ │ Penalty box │ Separate ipset/service → shield_penalty │ │ Allowlist │ Hardcoded in script → shield_allow ipset │ │ Dashboard │ BASH/iptables poll → Python delta packets │ │ Reports │ None → HTML threat analysis reports │ └──────────────────────────┴─────────────────────────────────-─────────┘
```

### The Delta Rebuild Advantage

The most impactful change. Instead of re-fetching everything nightly, the Python system tracks the last successful WHOIS lookup for every ASN and the ETag of every country block file:

```
Python nightly run cost (approximate): ASN WHOIS lookups (only stale, 14-day TTL) ~2-5 per night Country file downloads (only changed, ETag) 0-3 per night (usually 0) DB diff computation ~5 seconds ipset delta apply ~10 seconds Total rebuild time: ~30 seconds typical Total per week: ~3.5 minutes
```

That's roughly a 700× reduction in rebuild time under normal conditions.

## The Migration Process

The migration was executed in four phases with no loss of protection at any point.

```
Phase 1: Database Seeding ↓ Python DB populated while BASH system continues running Both systems coexist. Python touches no live ipsets. Phase 2: Shadow Mode (7+ days) ↓ Python rebuild runs with --dry-run flag Logs all changes it WOULD make, modifies nothing Compared against BASH output each morning Phase 3: Cutover ↓ BASH cron entries commented out (not deleted) Python cron entries go live New systemd boot services enabled 72-hour intensive monitoring period Phase 4: Cleanup (30 days later) ↓ BASH scripts archived to tar.gz Old flat files removed 502,575-line /etc/ipset.conf deleted Orphan ipsets destroyed Old systemd services masked
```

### What Was Decommissioned

The cutover removed a significant amount of legacy infrastructure:

- `/usr/local/bin/enterprise_shield.sh` (the main BASH script)
- `/usr/local/bin/block_asn.sh` (the injection tool)
- `/usr/local/bin/abuseipdb_penaltybox.sh`
- `/usr/local/bin/shield-stat.sh` and `shield_display.sh`
- `/etc/ipset.conf` — 502,575 lines, gone
- `shield-ipset-restore.service` (BASH-era)
- `shield-iptables-restore.service` (BASH-era)
- The orphan `SHIELD_PENALTY` ipset that had survived via `netfilter-persistent`

The new Python system replaced all of this with a clean installation under `/usr/local/lib/enterprise_shield/`, state in `/var/lib/enterprise_shield/shield.db`,
and configuration in `/etc/enterprise_shield/`.

## Key Decisions Made Along the Way

Several non-obvious architectural choices shaped the Python system. These are worth documenting because they affected behaviour:

**Hard DELETEs over soft deletes** — Earlier design drafts used a soft-delete pattern (marking rows inactive rather than removing them). This was rejected: it causes the database to grow indefinitely, and there's no useful recovery scenario where you'd
want to un-delete a stale CIDR entry. The Python system uses hard DELETEs.

**AbuseIPDB gets its own table** — The initial design merged AbuseIPDB IPs into the main `cidr_blocks` table. This was wrong: AbuseIPDB operates on a completely different lifecycle (5× daily refresh, IP-level not CIDR-level, confidence threshold filtering).
It got its own `abuseipdb_entries` table with independent process ownership.

**ip-api.com not BGPView** — Report enrichment (ASN name, country, city for attacker IPs) originally used BGPView. This was replaced with ip-api.com: 45 req/min free tier, no auth, returns ASN + org + country + city in a single call. Shodan's InternetDB
(`internetdb.shodan.io/{ip}`) was added for open port data and threat tags — also no auth.

**All runtime state in the database** — No flat files for parser position, public IP cache, or rebuild flags. The `system_state` table in SQLite holds everything. This means the database backup is the *complete* system backup.

**SHA-256 config change detection** — The rebuild checks a SHA-256 hash of the config files at startup. If anything changed since the last run, a full re-import of config is triggered automatically. No manual `--import` step needed.

## The Result

Enterprise Shield v12 reached verified production status on 2026-05-27, after extensive testing including multiple reboot verification cycles. All 16 items on the post-reboot checklist pass cleanly.

The system now:

- Rebuilds in ~30 seconds instead of ~38 minutes
- Has a complete history of every CIDR change
- Logs every firewall hit to a structured database
- Produces HTML threat analysis reports
- Survives reboots with a DB-driven restore rather than a flat-file restore
- Has crash recovery via the `rebuild_in_progress` flag
- Provides a real-time terminal dashboard (`shield_stat.py`) with delta-based packet rate bars

The database contains ~500,000 active CIDR entries across 7 ipsets, ~10,000 AbuseIPDB entries, and growing hit log data as the rsyslog pipeline feeds it.

The next post will go into the architecture of the system so that if you're considering implementing something similar, you can see how to configure the entire flow.