Over the years, I have tried to limit the amount of spurious or aggressive traffic against my small, yet mighty web server environment. It started with a list of UFW rules, but that became very unwieldy once the list hit over 25,000 unique CIDR blocks.
A few months ago, working alongside some AI partners, I asked how this system could be made more efficient and manage even larger rulesets without making the server scream in pain every time a large burst of traffic hit it.
That’s when the Enterprise Shield approach was born.
What Is Enterprise Shield?
Enterprise Shield is a custom Linux firewall management system protecting creaky2, a vintage 2008 MacBook Pro repurposed as a personal Ubuntu Server hosting performancezen.com. It is not a product — it is a purpose-built system, grown organically over years of dealing with real-world attacks against a small but publicly visible WordPress blog.
At its core, Enterprise Shield does three things:
- Blocks hostile IP space — autonomous systems (ASNs), country CIDR ranges, and individually flagged IPs — before they ever reach the web server.
- Rate-limits legitimate but noisy infrastructure — Microsoft Azure, AWS, Google Cloud, Oracle Cloud, Cloudflare — so they can’t hammer the server but aren’t outright blocked.
- Logs, enriches, and reports on everything that hits the firewall so you can understand the threat landscape over time.
The system wraps Linux’s ipset and iptables primitives with custom tooling to make them manageable: automating blocklist maintenance, surviving reboots, and providing a real-time terminal dashboard and HTML threat analysis reports.
A Brief History: How We Got Here
The Early Days: Hand-Crafted BASH (v11.x)
Enterprise Shield started as a BASH script. Over time it grew significantly:
- ASN blocking via RADB WHOIS lookups, parallelised across threads
- Country blocking via the ipverse/country-ip-blocks
GitHub feed (43 countries) - AbuseIPDB integration — a penalty box of individually-flagged malicious IPs, refreshed 5× daily (constrained by the free API tier)
- Azure and hyperscaler separation — Azure (
AS8075) given its own rate-limit chain distinct from AWS/GCP/Oracle/Cloudflare - Atomic ipset swaps — the live blocklist is never empty during an update
- Delta safety abort — if the new CIDR count drops >5% from the previous run, the script aborts rather than switching to a degraded ruleset
- Systemd boot persistence — two services that restore ipsets and iptables chains after every reboot
By version 11.25-MT, the script was around 500 lines, handled parallel WHOIS with a 4-thread batch model, and managed a blocklist of over 500,000 CIDR entries.
It worked. But it had accumulated significant technical debt.
Why We Rewrote It
Problem 1: Every Run Was a Full Rebuild
The BASH system had no memory between runs. Every night at 2AM:
- All ASN WHOIS data was re-fetched — 447 ASNs × a RADB query each, regardless of whether anything had changed
- All 43 country CIDR files were re-downloaded from GitHub
- All 500,000+ CIDR entries were re-inserted from scratch into a staging ipset
This wasn’t just slow — it was wasteful. ASN CIDR allocations change rarely. The ipverse country files change weekly at most. But the system had no way to say “I already know what AS3209 owns — skip it.”
BASH nightly run cost (approximate):
447 ASN WHOIS lookups ~35 minutes
43 GitHub downloads ~2 minutes
ipset full rebuild ~1 minute
Total rebuild time: ~38 minutes
Total per week: ~4.5 hours of server timeProblem 2: State Was Scattered Across Flat Files
Runtime state was stored in flat files:
/etc/ipset.conf— the full ipset dump (502,575 lines)/etc/shield-iptables.rules— the iptables chain rules/var/lib/shield/last_entry_count— just a number, for the delta check/var/lib/shield/last_public_ip— the server’s external IP at last run
There was no audit trail, no history, no way to answer “what changed between last Tuesday and today?” The last_entry_count file was a single integer — useful for
the delta abort check, useless for anything else.
Problem 3: No Hit Data
The BASH system blocked things — but didn’t know what it blocked. There was no structured record of which IPs hit the firewall, when, or how many times. The iptables -L packet counters reset on every rebuild. A threat analysis report
was impossible.
Problem 4: AbuseIPDB Was Bolted On
The AbuseIPDB penalty box was a separate script (abuseipdb_penaltybox.sh) that maintained its own ipset (SHIELD_PENALTY), with its own boot persistence service,
its own log file, and completely separate state from the main system. It worked, but it was an island.
Problem 5: Boot Persistence Was Fragile
The two-service boot sequence (ipsets before UFW, chains after UFW) worked correctly but relied on /etc/ipset.conf — a 502,575-line flat file. If that file was corrupt or missing, the server came up with no firewall. There was no crash recovery, no verification that the restored state was sane.
The Solution: Python + SQLite
The rewrite was designed around a single guiding principle:
The database is the system. Everything else derives from it.
What Changed
┌────────────────────────────────────────────────────────────-─────────┐
│ BASH v11.25-MT → Python v12 │
├──────────────────────────┬────────────────────────────────-──────────┤
│ State storage │ Flat files → SQLite database │
│ Rebuild strategy │ Full re-fetch → Delta (only changed data) │
│ WHOIS cadence │ Every night for all ASNs → Only stale │
│ Country blocks │ Re-download always → ETag conditional GET │
│ AbuseIPDB │ Separate script/ipset → Integrated table │
│ Hit logging │ None → rsyslog → hits_parser → SQLite │
│ Boot persistence │ Flat file restore → DB-driven restore.py │
│ Crash recovery │ None → rebuild_in_progress flag in DB │
│ Config change detection │ None → SHA-256 hash of config files │
│ Penalty box │ Separate ipset/service → shield_penalty │
│ Allowlist │ Hardcoded in script → shield_allow ipset │
│ Dashboard │ BASH/iptables poll → Python delta packets │
│ Reports │ None → HTML threat analysis reports │
└──────────────────────────┴─────────────────────────────────-─────────┘The Delta Rebuild Advantage
The most impactful change. Instead of re-fetching everything nightly, the Python system tracks the last successful WHOIS lookup for every ASN and the ETag of every country block file:
Python nightly run cost (approximate):
ASN WHOIS lookups (only stale, 14-day TTL) ~2-5 per night
Country file downloads (only changed, ETag) 0-3 per night (usually 0)
DB diff computation ~5 seconds
ipset delta apply ~10 seconds
Total rebuild time: ~30 seconds typical
Total per week: ~3.5 minutesThat’s roughly a 700× reduction in rebuild time under normal conditions.
The Migration Process
The migration was executed in four phases with no loss of protection at any point.
Phase 1: Database Seeding
↓
Python DB populated while BASH system continues running
Both systems coexist. Python touches no live ipsets.
Phase 2: Shadow Mode (7+ days)
↓
Python rebuild runs with --dry-run flag
Logs all changes it WOULD make, modifies nothing
Compared against BASH output each morning
Phase 3: Cutover
↓
BASH cron entries commented out (not deleted)
Python cron entries go live
New systemd boot services enabled
72-hour intensive monitoring period
Phase 4: Cleanup (30 days later)
↓
BASH scripts archived to tar.gz
Old flat files removed
502,575-line /etc/ipset.conf deleted
Orphan ipsets destroyed
Old systemd services maskedWhat Was Decommissioned
The cutover removed a significant amount of legacy infrastructure:
/usr/local/bin/enterprise_shield.sh(the main BASH script)/usr/local/bin/block_asn.sh(the injection tool)/usr/local/bin/abuseipdb_penaltybox.sh/usr/local/bin/shield-stat.shandshield_display.sh/etc/ipset.conf— 502,575 lines, goneshield-ipset-restore.service(BASH-era)shield-iptables-restore.service(BASH-era)- The orphan
SHIELD_PENALTYipset that had survived vianetfilter-persistent
The new Python system replaced all of this with a clean installation under /usr/local/lib/enterprise_shield/, state in /var/lib/enterprise_shield/shield.db,
and configuration in /etc/enterprise_shield/.
Key Decisions Made Along the Way
Several non-obvious architectural choices shaped the Python system. These are worth documenting because they affected behaviour:
Hard DELETEs over soft deletes — Earlier design drafts used a soft-delete pattern (marking rows inactive rather than removing them). This was rejected: it causes the database to grow indefinitely, and there’s no useful recovery scenario where you’d
want to un-delete a stale CIDR entry. The Python system uses hard DELETEs.
AbuseIPDB gets its own table — The initial design merged AbuseIPDB IPs into the main cidr_blocks table. This was wrong: AbuseIPDB operates on a completely different lifecycle (5× daily refresh, IP-level not CIDR-level, confidence threshold filtering).
It got its own abuseipdb_entries table with independent process ownership.
ip-api.com not BGPView — Report enrichment (ASN name, country, city for attacker IPs) originally used BGPView. This was replaced with ip-api.com: 45 req/min free tier, no auth, returns ASN + org + country + city in a single call. Shodan’s InternetDB
(internetdb.shodan.io/{ip}) was added for open port data and threat tags — also no auth.
All runtime state in the database — No flat files for parser position, public IP cache, or rebuild flags. The system_state table in SQLite holds everything. This means the database backup is the complete system backup.
SHA-256 config change detection — The rebuild checks a SHA-256 hash of the config files at startup. If anything changed since the last run, a full re-import of config is triggered automatically. No manual --import step needed.
The Result
Enterprise Shield v12 reached verified production status on 2026-05-27, after extensive testing including multiple reboot verification cycles. All 16 items on the post-reboot checklist pass cleanly.
The system now:
- Rebuilds in ~30 seconds instead of ~38 minutes
- Has a complete history of every CIDR change
- Logs every firewall hit to a structured database
- Produces HTML threat analysis reports
- Survives reboots with a DB-driven restore rather than a flat-file restore
- Has crash recovery via the
rebuild_in_progressflag - Provides a real-time terminal dashboard (
shield_stat.py) with delta-based packet rate bars
The database contains ~500,000 active CIDR entries across 7 ipsets, ~10,000 AbuseIPDB entries, and growing hit log data as the rsyslog pipeline feeds it.
The next post will go into the architecture of the system so that if you’re considering implementing something similar, you can see how to configure the entire flow.
Leave a Reply