Skip to content
This repository was archived by the owner on Apr 14, 2025. It is now read-only.
This repository was archived by the owner on Apr 14, 2025. It is now read-only.

Possible fail2ban regression in lat update causing high CPU usage #54

@vhsantos

Description

@vhsantos

Hi everyone,

I suspect that the latest update (4fd791c) introduced a regression.

Since the update, my CPU usage has been at 100% almost constantly. The issue appears to be related to fail2ban, which adds IPs to ipset, but then the fail2ban service becomes unresponsive for a moment. As a result, all IPs are removed, and fail2ban starts over, repeatedly adding them again.

I linked this issue to the update because my monitoring graphs show that CPU usage spiked right after 02:30 AM, which is when the update was applied.

Image

Here some outputs when it is "working":

# fail2ban-client status mc-exim-1m
Status for the jail: mc-exim-1m
|- filter
|  |- File list:	/var/mailcleaner/log/exim_stage1/rejectlog 
|  |- Currently failed:	9
|  `- Total failed:	57284
`- action
   |- Currently banned:	1026
   |  `- IP list:	here all the IPs bloqued
   `- Total banned:	1026

then:

fail2ban-client status mc-exim-1m
ERROR  Unable to contact server. Is it running?

maybe 30-90 seconds later:

 fail2ban-client status mc-exim-1m
Status for the jail: mc-exim-1m
|- filter
|  |- File list:	/var/mailcleaner/log/exim_stage1/rejectlog 
|  |- Currently failed:	10
|  `- Total failed:	4785
`- action
   |- Currently banned:	47
   |  `- IP list:	here all the IPs bloqued
   `- Total banned:	47

I have looked on the kernel/syslog logs to check for a possible OOM, but nothing..
on the fail2ban.log nothing except this,

2025-03-04 16:14:13,839 fail2ban.actions.action[1629]: ERROR   ipset destroy mc-ssh-1y returned 100
2025-03-04 16:14:14,834 fail2ban.actions.action[1629]: ERROR   ipset destroy mc-webauth-1d returned 100
2025-03-04 16:14:15,819 fail2ban.actions.action[1629]: ERROR   ipset destroy mc-exim-1d returned 100
2025-03-04 16:14:15,834 fail2ban.actions.action[1629]: ERROR   ipset destroy mc-webauth-1m returned 100
2025-03-04 16:14:16,827 fail2ban.actions.action[1629]: ERROR   ipset destroy mc-exim-1m returned 100
2025-03-04 16:14:16,832 fail2ban.actions.action[1629]: ERROR   ipset destroy mc-ssh-1w returned 100
2025-03-04 16:14:16,838 fail2ban.actions.action[1629]: ERROR   ipset destroy mc-webauth-1w returned 100
2025-03-04 16:14:17,835 fail2ban.actions.action[1629]: ERROR   ipset destroy mc-ssh-1m returned 100
2025-03-04 16:14:20,569 fail2ban.actions.action[2455]: ERROR   ipset create mc-exim-1d hash:ip returned 100
2025-03-04 16:14:20,572 fail2ban.actions.action[2455]: ERROR   ipset create mc-exim-1m hash:ip returned 100
2025-03-04 16:14:20,577 fail2ban.actions.action[2455]: ERROR   ipset create mc-exim-1w hash:ip returned 100
2025-03-04 16:14:20,581 fail2ban.actions.action[2455]: ERROR   ipset create mc-exim-1y hash:ip returned 100
2025-03-04 16:14:20,584 fail2ban.actions.action[2455]: ERROR   ipset create mc-ssh-1d hash:ip returned 100
2025-03-04 16:14:20,588 fail2ban.actions.action[2455]: ERROR   ipset create mc-ssh-1m hash:ip returned 100
2025-03-04 16:14:20,592 fail2ban.actions.action[2455]: ERROR   ipset create mc-ssh-1w hash:ip returned 100
2025-03-04 16:14:20,602 fail2ban.actions.action[2455]: ERROR   ipset create mc-ssh-1y hash:ip returned 100
2025-03-04 16:14:20,608 fail2ban.actions.action[2455]: ERROR   ipset create mc-webauth-1d hash:ip returned 100
2025-03-04 16:14:20,612 fail2ban.actions.action[2455]: ERROR   ipset create mc-webauth-1m hash:ip returned 100
2025-03-04 16:14:20,614 fail2ban.actions.action[2455]: ERROR   ipset create mc-webauth-1w hash:ip returned 100
2025-03-04 16:14:20,616 fail2ban.actions.action[2455]: ERROR   ipset create mc-webauth-1y hash:ip returned 100
2025-03-04 16:15:02,656 fail2ban.filter [2455]: ERROR   Error in FilterPyinotify callback: 'module' object has no attribute '_strptime_time'
2025-03-04 16:15:02,657 fail2ban.filter [2455]: ERROR   Error in FilterPyinotify callback: 'module' object has no attribute '_strptime_time'
2025-03-04 16:15:02,657 fail2ban.filter [2455]: ERROR   Error in FilterPyinotify callback: 'module' object has no attribute '_strptime_time'
2025-03-04 17:00:18,003 fail2ban.actions.action[64104]: ERROR   ipset create mc-exim-1d hash:ip returned 7f00
2025-03-04 17:00:18,007 fail2ban.actions.action[64104]: ERROR   ipset create mc-exim-1m hash:ip returned 7f00
2025-03-04 17:00:18,012 fail2ban.actions.action[64104]: ERROR   ipset create mc-exim-1w hash:ip returned 7f00
2025-03-04 17:00:18,020 fail2ban.actions.action[64104]: ERROR   ipset create mc-exim-1y hash:ip returned 7f00
2025-03-04 17:00:18,023 fail2ban.actions.action[64104]: ERROR   ipset create mc-ssh-1d hash:ip returned 7f00
2025-03-04 17:00:18,026 fail2ban.actions.action[64104]: ERROR   ipset create mc-ssh-1m hash:ip returned 7f00
2025-03-04 17:00:18,029 fail2ban.actions.action[64104]: ERROR   ipset create mc-ssh-1w hash:ip returned 7f00
2025-03-04 17:00:18,032 fail2ban.actions.action[64104]: ERROR   ipset create mc-ssh-1y hash:ip returned 7f00
2025-03-04 17:00:18,035 fail2ban.actions.action[64104]: ERROR   ipset create mc-webauth-1d hash:ip returned 7f00
2025-03-04 17:00:18,038 fail2ban.actions.action[64104]: ERROR   ipset create mc-webauth-1m hash:ip returned 7f00
2025-03-04 17:00:18,057 fail2ban.actions.action[64104]: ERROR   ipset create mc-webauth-1w hash:ip returned 7f00
2025-03-04 17:00:18,058 fail2ban.actions.action[64104]: ERROR   ipset create mc-webauth-1y hash:ip returned 7f00
2025-03-04 17:15:17,259 fail2ban.actions.action[49981]: ERROR   ipset create mc-exim-1d hash:ip returned 7f00
2025-03-04 17:15:17,265 fail2ban.actions.action[49981]: ERROR   ipset create mc-exim-1m hash:ip returned 7f00
2025-03-04 17:15:17,271 fail2ban.actions.action[49981]: ERROR   ipset create mc-exim-1w hash:ip returned 7f00
2025-03-04 17:15:17,279 fail2ban.actions.action[49981]: ERROR   ipset create mc-exim-1y hash:ip returned 7f00
2025-03-04 17:15:17,282 fail2ban.actions.action[49981]: ERROR   ipset create mc-ssh-1d hash:ip returned 7f00
2025-03-04 17:15:17,287 fail2ban.actions.action[49981]: ERROR   ipset create mc-ssh-1m hash:ip returned 7f00
2025-03-04 17:15:17,293 fail2ban.actions.action[49981]: ERROR   ipset create mc-ssh-1w hash:ip returned 7f00
2025-03-04 17:15:17,299 fail2ban.actions.action[49981]: ERROR   ipset create mc-ssh-1y hash:ip returned 7f00
2025-03-04 17:15:17,306 fail2ban.actions.action[49981]: ERROR   ipset create mc-webauth-1d hash:ip returned 7f00
2025-03-04 17:15:17,309 fail2ban.actions.action[49981]: ERROR   ipset create mc-webauth-1m hash:ip returned 7f00
2025-03-04 17:15:17,313 fail2ban.actions.action[49981]: ERROR   ipset create mc-webauth-1w hash:ip returned 7f00
2025-03-04 17:15:17,324 fail2ban.actions.action[49981]: ERROR   ipset create mc-webauth-1y hash:ip returned 7f00
2025-03-04 17:15:28,941 fail2ban.filter [49981]: ERROR   Error in FilterPyinotify callback: 'module' object has no attribute '_strptime_time'
2025-03-04 17:15:28,942 fail2ban.filter [49981]: ERROR   Error in FilterPyinotify callback: 'module' object has no attribute '_strptime_time'
2025-03-04 17:15:28,942 fail2ban.filter [49981]: ERROR   Error in FilterPyinotify callback: 'module' object has no attribute '_strptime_time'
2025-03-04 17:30:19,231 fail2ban.actions.action[45650]: ERROR   ipset create mc-exim-1d hash:ip returned 7f00
2025-03-04 17:30:19,238 fail2ban.actions.action[45650]: ERROR   ipset create mc-exim-1m hash:ip returned 7f00
2025-03-04 17:30:19,241 fail2ban.actions.action[45650]: ERROR   ipset create mc-exim-1w hash:ip returned 7f00
2025-03-04 17:30:19,247 fail2ban.actions.action[45650]: ERROR   ipset create mc-exim-1y hash:ip returned 7f00
2025-03-04 17:30:19,251 fail2ban.actions.action[45650]: ERROR   ipset create mc-ssh-1d hash:ip returned 7f00
2025-03-04 17:30:19,259 fail2ban.actions.action[45650]: ERROR   ipset create mc-ssh-1m hash:ip returned 7f00
2025-03-04 17:30:19,262 fail2ban.actions.action[45650]: ERROR   ipset create mc-ssh-1w hash:ip returned 7f00
2025-03-04 17:30:19,267 fail2ban.actions.action[45650]: ERROR   ipset create mc-ssh-1y hash:ip returned 7f00
2025-03-04 17:30:19,274 fail2ban.actions.action[45650]: ERROR   ipset create mc-webauth-1d hash:ip returned 7f00
2025-03-04 17:30:19,277 fail2ban.actions.action[45650]: ERROR   ipset create mc-webauth-1m hash:ip returned 7f00
2025-03-04 17:30:19,280 fail2ban.actions.action[45650]: ERROR   ipset create mc-webauth-1w hash:ip returned 7f00
2025-03-04 17:30:19,293 fail2ban.actions.action[45650]: ERROR   ipset create mc-webauth-1y hash:ip returned 7f00
2025-03-04 17:30:30,994 fail2ban.filter [45650]: ERROR   Error in FilterPyinotify callback: 'module' object has no attribute '_strptime_time'
2025-03-04 17:30:30,994 fail2ban.filter [45650]: ERROR   Error in FilterPyinotify callback: 'module' object has no attribute '_strptime_time'
2025-03-04 17:30:30,995 fail2ban.filter [45650]: ERROR   Error in FilterPyinotify callback: 'module' object has no attribute '_strptime_time'

Putting the fail2ban in debug mode (loglevel 4), show nothing outside of normal neither.

Any idea or suggestions ??

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions