Bulk AbuseIPDB reporting using command-line tools

Some botnet is scraping my local Gitea instance by requesting every single URL possible. Even the same table is queried in every variation possible: sorted ascending, sorted descending, sorted by second column ascending, sorted by second column descending, and so on. This creates unnecessary load on the server and traffic on the line.

Almost every single request is coming from a random IP address from all over the world. Also, they’re all using randomised User-Agent strings – many of which don’t even exist. A few years ago, this would have been classed as a DDoS attack.

A user on Hacker News pointed me to https://www.abuseipdb.com/ – a website similar to spamcop.net, but where you can report IP addresses instead of email addresses.

With a little command-line magic, I was able to create a CSV file for their Bulk Reporter tool:

echo "IP,Categories,ReportDate,Comment" > /tmp/abuseipdb.csv
cat access.log \
| grep " 302 " | grep "sort=" \
| awk '{print $1",\
\"19\",\
\""substr($4, 2), substr($5, 0, length($5)-1)"\",\
\"Distributed parallel HTTP scraping from random IPs with random user-agents\""}' \
| sort -t, -k1,1 -u >> /tmp/abuseipdb.csvCode language: Bash (bash)

This filters my Traefik access.log for lines with a HTTP 302 redirect (which I’ve setup temporarily – however, the bots are still requesting URLs that aren’t there for 3 days now) and sort= in the URL – which is the parameter for the mentioned sort variations.

Once filtered, awk prints the first field (IP address), followed by "19" which is AbuseIPDB’s code for Bad Web Bot. It then cuts 2 characters off the fourth field followed by the fifth field with the last character cut off. Traefik logs time stamps in the form [30/Aug/2026:20:00:00 +0000] and awk sees this with a leading space, so this makes sure the square brackets are gone.

(Also, I’ve tested that the time string is properly parsed by strtotime().)

Finally it’s putting out the comment/reason for the report.

This is then run through sort with a comma as separator and field 1 (and only field 1) as the key to be made unique, i.e. duplicate IP addresses will be omitted.

The final result is then output into the file /tmp/abuseipdb.csv – ready to be uploaded to the site. Or pushed via their API:

curl https://api.abuseipdb.com/api/v2/bulk-report -F csv=@/tmp/abuseipdb.csv -H "Key: API_KEY"Code language: JavaScript (javascript)

JSON instead of CLF

I’ve recently switched my Traefik to JSON logs so I have more details like requested hostname and HTTP headers. To prepare this for submission, the tool jq comes in handy:

echo "IP,Categories,ReportDate,Comment" > /tmp/abuseipdb.csv
jq -r 'select(.DownstreamStatus == 302)
        | select((.RequestPath | contains("sort=")) or (.RequestPath | contains("blame")))
        | [.ClientHost, "19", .time, "Distributed parallel HTTP scraping from random IPs with random user-agents"]
        | @csv' access.log \
    | tail -n 5000 \
    | sort -t, -k1,1 -u >> /tmp/abuseipdb.csvCode language: Bash (bash)

This selects log entries with HTTP 302 and either sort= or blame in their requested URL. It then only takes the latest 5000 requests (my daily report limit at AbuseIPDB), makes sure the IPs are unique and writes the CSV file.