Sometimes not unbanning

[pjv]

Hi @jkbzh, thanks for this code - saved me writing something like it.

After setting everything up and testing and seemingly getting it working fine, I'm noticing that sometimes IP's are successfully getting unbanned (by f2b) and sometimes they are not. This results in the temp ban IP list growing and containing old bans that should have timed out and been deleted.

I know that it's an intermittent issue because i set up some "unban" logging and when I went through the log of unban requests and tried to manually match them against the CF list, some were there and some weren't.

I am looking for a way to try to debug this issue and wondering if you have any suggestions. Do you know where I might look for logs (running under ubuntu)?

I was looking at the https://github.com/cloudflare/cloudflare-python repo's readme and thought maybe adjusting the "retries" parameter on the list_items_delete_by_id function could possibly help.

I also noticed that the cloudflare-python is (currently) at version 4.3.1 but f2b_cf is locked at 3.1.1 and was wondering about trying to update the cloudflare module. Any thoughts on that?

[pjv]

update: I was able to find some logging in the fail2ban log. <bad ipv4 address> is one of several addresses i found in my CF temp ban list that should have been unbanned but wasn't.

2025-09-07 10:53:45,857 fail2ban.filter         [25375]: INFO    [wordpress-hard] Found <bad ipv4 address> - 2025-09-07 10:53:45
2025-09-07 10:53:45,857 fail2ban.filter         [25375]: INFO    [wordpress-hard] Found <bad ipv4 address> - 2025-09-07 10:53:45
2025-09-07 10:53:46,057 fail2ban.actions        [25375]: NOTICE  [wordpress-hard] Ban <bad ipv4 address>
2025-09-07 12:53:48,398 fail2ban.actions        [25375]: NOTICE  [wordpress-hard] Unban <bad ipv4 address>
2025-09-07 12:53:51,079 fail2ban.utils          [25375]: ERROR   7f665469de30 -- exec: /usr/local/bin/f2b_cf_plx --unban <bad ipv4 address> && /usr/local/bin/f2b_cf_lbk --unban <bad ipv4 address>
2025-09-07 12:53:51,079 fail2ban.utils          [25375]: ERROR   7f665469de30 -- stderr: 'error finding list item for ip <bad ipv4 address>'
2025-09-07 12:53:51,080 fail2ban.actions        [25375]: ERROR   Failed to execute unban jail 'wordpress-hard' action 'f2b-cf-temp-ban' info 'ActionInfo({'ip': '<bad ipv4 address>', 'family': 'inet4', 'fid': <function Actions.ActionInfo.<lambda> at 0x7f665cfbda20>, 'raw-ticket': <function Actions.ActionInfo.<lambda> at 0x7f665cfbe0e0>})': Error unbanning <bad ipv4 address>

I'm doing some testing now to nail down why / how list_items_delete_by_ip is throwing stderr: 'error finding list item for ip <bad ipv4 address>'

(note: i maintain two separate temp ban ip lists with the same contents in two separate CF accounts, hence the && in the exec line of the log)

[pjv]

IGNORE THIS (see next comment below)

---

OK, so i have a kludgy fix, but it seems like a working fix in my setup.

I couldn't find anything in the code that wasn't working but i think that the CF API sometimes returns flaky results for client.rules.lists.items.list. That call is in the function list_items_list in f2b_cf.py.

CF isn't returning an error or a null, it seems that it sometimes returns an incomplete or maybe empty list and because the target ip is not in the list, then list_items_get_by_ip fails to return the item's id and then list_items_delete_by_ip fails with f"error finding list item for ip {ip}".

My kludgy fix is to just throw a couple retries into list_items_delete_by_ip with a sleep. In my case this is fixing the unban problem and keeping my temp ban list in perfect sync with fail2ban. I'll send you a PR if you want but this is so kludgy that I'm just gonna show the code here first and you can tell me if you want to use it as-is or whatever.

first add a new import:
from time import sleep

then:

def list_items_delete_by_ip(client, account_id, list_id, ip):
  "deletes an item with ip 'ip' from an IP list"

  item_id = list_items_get_by_ip(client, account_id, list_id, ip)

  if not item_id:
    sleep(5)
    item_id = list_items_get_by_ip(client, account_id, list_id, ip)
    if not item_id:
      sleep(5)
      item_id = list_items_get_by_ip(client, account_id, list_id, ip)
      if not item_id:
        raise Fail2BanCloudflareError(f"error finding list item for ip {ip}")

  return list_items_delete_by_id(client, account_id, list_id, item_id)

[pjv]

I have a new causal hypothesis for what may be happening here. There are a number of issues on the cloudflare-python repo that are all some version of the CF list auto-pagination being broken and only returning the first page of results:

cloudflare/cloudflare-python#2667
cloudflare/cloudflare-python#2584
cloudflare/cloudflare-python#2576

If that is the cause of the failure to find a given IP in the list, then my kludgy fix above won't work and i've just been luckily not having my temp-ban list grow to beyond the length that is returned in the first page in the time i've been testing the fix.

Edit: CONFIRMED that faulty pagination is indeed the issue.

if you perform a dump (f2b_cf -d) on a list with more than 25 items, it will correctly tell you how many items there are, but it will only print 25 items.

Here is a new kludgy work-around. In the function list_items_list, add a per_page parameter as in the below. Note that the maximum number you can supply is 500, so if a list has more than 500 items this kludge will fail.

def list_items_list(client, account_id, list_id):
  "get list items (returns a list)"

  rv = client.rules.lists.items.list(
    account_id=account_id, list_id=list_id, per_page=500
  )
  if not rv or not rv.success:
    raise Fail2BanCloudflareError(f"error accessing list {list_id}")
  return rv