[pfsense_rule] Duplicate TrackingID in rule being created.

[Inocious]

Describe the bug
When using the below task, duplicates TrackingID rules are being created. It's then unable to track effectively anything in FW logs. In the logs here, it's TID 1765964518.
I have this simple task that creates 4 rules below under my MGMT interface. But all interface are concerned with this issue.
My target interface is named MGMT, one alias named Dell_Ethernet is used.

- name: "Allow MGMT subnet to ping *"
  pfsensible.core.pfsense_rule:
    name: 'Allow MGMT subnet to ping *'
    action: pass
    log: true
    interface: mgmt
    ipprotocol: inet
    protocol: icmp
    icmptype: any
    source: NET:MGMT
    destination: any
    after: top
    state: present
    disabled: true

- name: "Allow any to any"
  pfsensible.core.pfsense_rule:
    name: 'Allow any to any'
    action: pass
    log: true
    interface: mgmt
    ipprotocol: inet
    protocol: any
    destination: any
    source: any
    after: 'Allow MGMT subnet to ping *'
    state: present
    disabled: true

- name: "Allow Hosts to connect to pfsense"
  pfsensible.core.pfsense_rule:
    name: 'Allow Hosts to connect to pfsense'
    action: pass
    log: true
    interface: mgmt
    ipprotocol: inet
    protocol: tcp
    source: Dell_Ethernet
    destination: IP:MGMT
    destination_port: 80
    after: 'Allow any to any'
    state: present

- name: "Default: Block all"
  pfsensible.core.pfsense_rule:
    name: 'Default: Block all'
    action: block
    log: true
    interface: mgmt
    ipprotocol: inet
    protocol: any
    destination: any
    source: any
    after: 'Allow Hosts to connect to pfsense'
    state: present

Output
Please paste the ansible output run with -vv:

TASK [conf_pfsense : Allow MGMT subnet to ping *] ********************************************************************************************************************************************************************************************************************
task path: configure_rules_MGMT.yml:1
changed: [pfsenseAnsible] => {"added": [{"created": {"time": "1765964517", "username": "root@<ANSIBLE_CONTROLLER_IP_REDACTED> 48960 22"}, "descr": "Allow MGMT subnet to ping *", "destination": {"any": null}, "disabled": "", "icmptype": "any", "id": "", "interface": "opt1", "ipprotocol": "inet", "log": "", "protocol": "icmp", "source": {"network": "opt1"}, "statetype": "keep state", "tracker": "1765964517", "type": "pass", "updated": {"time": "1765964517", "username": "root@<ANSIBLE_CONTROLLER_IP_REDACTED> 48960 22"}}], "changed": true, "commands": ["create rule 'Allow MGMT subnet to ping *' on 'mgmt', source='NET:MGMT', destination='any', protocol='icmp', disabled=True, log=True, after='top'"], "deleted": [], "modified": [], "stderr": "", "stderr_lines": [], "stdout": "pfSense shell: global $debug;\npfSense shell: $debug = 1;\npfSense shell: require_once(\"filter.inc\");\npfSense shell: \r<figure() == 0) { clear_subsystem_dirty('filter'); }\npfSense shell: exec\npfSense shell: exit\n", "stdout_lines": ["pfSense shell: global $debug;", "pfSense shell: $debug = 1;", "pfSense shell: require_once(\"filter.inc\");", "pfSense shell: ", "<figure() == 0) { clear_subsystem_dirty('filter'); }", "pfSense shell: exec", "pfSense shell: exit"]}

TASK [conf_pfsense : Allow any to any] *******************************************************************************************************************************************************************************************************************************
task path: configure_rules_MGMT.yml:16
changed: [pfsenseAnsible] => {"added": [{"created": {"time": "1765964518", "username": "root@<ANSIBLE_CONTROLLER_IP_REDACTED> 48966 22"}, "descr": "Allow any to any", "destination": {"any": null}, "disabled": "", "id": "", "interface": "opt1", "ipprotocol": "inet", "log": "", "source": {"any": null}, "statetype": "keep state", "tracker": "1765964518", "type": "pass", "updated": {"time": "1765964518", "username": "root@<ANSIBLE_CONTROLLER_IP_REDACTED> 48966 22"}}], "changed": true, "commands": ["create rule 'Allow any to any' on 'mgmt', source='any', destination='any', disabled=True, log=True, after='Allow MGMT subnet to ping *'"], "deleted": [], "modified": [], "stderr": "", "stderr_lines": [], "stdout": "pfSense shell: global $debug;\npfSense shell: $debug = 1;\npfSense shell: require_once(\"filter.inc\");\npfSense shell: \r<figure() == 0) { clear_subsystem_dirty('filter'); }\npfSense shell: exec\npfSense shell: exit\n", "stdout_lines": ["pfSense shell: global $debug;", "pfSense shell: $debug = 1;", "pfSense shell: require_once(\"filter.inc\");", "pfSense shell: ", "<figure() == 0) { clear_subsystem_dirty('filter'); }", "pfSense shell: exec", "pfSense shell: exit"]}

TASK [conf_pfsense : Allow Hosts to connect to pfsense] **************************************************************************************************************************************************************************************************************
task path: configure_rules_MGMT.yml:30
changed: [pfsenseAnsible] => {"added": [{"created": {"time": "1765964518", "username": "root@<ANSIBLE_CONTROLLER_IP_REDACTED> 48976 22"}, "descr": "Allow Hosts to connect to pfsense", "destination": {"network": "opt1ip", "port": "80"}, "id": "", "interface": "opt1", "ipprotocol": "inet", "log": "", "protocol": "tcp", "source": {"address": "Dell_Ethernet"}, "statetype": "keep state", "tracker": "1765964518", "type": "pass", "updated": {"time": "1765964518", "username": "root@<ANSIBLE_CONTROLLER_IP_REDACTED> 48976 22"}}], "changed": true, "commands": ["create rule 'Allow Hosts to connect to pfsense' on 'mgmt', source='Dell_Ethernet', destination='IP:MGMT', destination_port='80', protocol='tcp', log=True, after='Allow any to any'"], "deleted": [], "modified": [], "stderr": "", "stderr_lines": [], "stdout": "pfSense shell: global $debug;\npfSense shell: $debug = 1;\npfSense shell: require_once(\"filter.inc\");\npfSense shell: \r<figure() == 0) { clear_subsystem_dirty('filter'); }\npfSense shell: exec\npfSense shell: exit\n", "stdout_lines": ["pfSense shell: global $debug;", "pfSense shell: $debug = 1;", "pfSense shell: require_once(\"filter.inc\");", "pfSense shell: ", "<figure() == 0) { clear_subsystem_dirty('filter'); }", "pfSense shell: exec", "pfSense shell: exit"]}

TASK [conf_pfsense : Default: Block all] *****************************************************************************************************************************************************************************************************************************
task path: configure_rules_MGMT.yml:44
changed: [pfsenseAnsible] => {"added": [{"created": {"time": "1765964519", "username": "root@<ANSIBLE_CONTROLLER_IP_REDACTED> 48988 22"}, "descr": "Default: Block all", "destination": {"any": null}, "id": "", "interface": "opt1", "ipprotocol": "inet", "log": "", "source": {"any": null}, "statetype": "keep state", "tracker": "1765964519", "type": "block", "updated": {"time": "1765964519", "username": "root@<ANSIBLE_CONTROLLER_IP_REDACTED> 48988 22"}}], "changed": true, "commands": ["create rule 'Default: Block all' on 'mgmt', source='any', destination='any', action='block', log=True, after='Allow Hosts to connect to pfsense'"], "deleted": [], "modified": [], "stderr": "", "stderr_lines": [], "stdout": "pfSense shell: global $debug;\npfSense shell: $debug = 1;\npfSense shell: require_once(\"filter.inc\");\npfSense shell: \r<figure() == 0) { clear_subsystem_dirty('filter'); }\npfSense shell: exec\npfSense shell: exit\n", "stdout_lines": ["pfSense shell: global $debug;", "pfSense shell: $debug = 1;", "pfSense shell: require_once(\"filter.inc\");", "pfSense shell: ", "<figure() == 0) { clear_subsystem_dirty('filter'); }", "pfSense shell: exec", "pfSense shell: exit"]}

Issue seems to be that TrackingID is based on Time.
Somehow, created.time value can found to be the exact same as another rule.

We can see here (logs taken and beautify from above) Duplicate Tracking id on rule Allow any to any and Allow Hosts to connect to pfsense. Notice that created.time = tracker = updated.time for both rules :

{
	"added": [
		{
			"created": {
				"time": "1765964518",
				"username": "root@<ANSIBLE_CONTROLLER_IP_REDACTED> 48966 22"
			},
			"descr": "Allow any to any",
			"destination": {
				"any": null
			},
			"disabled": "",
			"id": "",
			"interface": "opt1",
			"ipprotocol": "inet",
			"log": "",
			"source": {
				"any": null
			},
			"statetype": "keep state",
			"tracker": "1765964518",
			"type": "pass",
			"updated": {
				"time": "1765964518",
				"username": "root@<ANSIBLE_CONTROLLER_IP_REDACTED> 48966 22"
			}
		}
	],
	"changed": true,
	"commands": [
		"create rule 'Allow any to any' on 'mgmt', source='any', destination='any', disabled=True, log=True, after='Allow MGMT subnet to ping *'"
	],
	"deleted": [],
	"modified": [],
	"stderr": "",
	"stderr_lines": [],
	"stdout": "pfSense shell: global $debug;\npfSense shell: $debug = 1;\npfSense shell: require_once(\"filter.inc\");\npfSense shell: \r<figure() == 0) { clear_subsystem_dirty('filter'); }\npfSense shell: exec\npfSense shell: exit\n",
	"stdout_lines": [
		"pfSense shell: global $debug;",
		"pfSense shell: $debug = 1;",
		"pfSense shell: require_once(\"filter.inc\");",
		"pfSense shell: ",
		"<figure() == 0) { clear_subsystem_dirty('filter'); }",
		"pfSense shell: exec",
		"pfSense shell: exit"
	]
}

Shouldn't happen.

Environment

• What version of pfsensible.core? pfsensible.core 0.6.2
• What version of ansible? core 2.19.4
• What version of pfSense? 2.7.2-RELEASE

[Inocious]

Issue got found. Check of existing TrackerID is not being performed correctly: self.root_elt.findall('tracker') in _check_tracker function. It returns always an empty array. As <tracker> is not a direct element of our ET object root_elt, it is inside always a <rule> so it fails to find it. We should use self.root_elt.findall('rule/tracker') or more globally but time consuming XPath instead: self.root_elt.findall('.//tracker') to fix the issue .

How I spot the issue 😃 :

In plugins/module_utils/rule.py at init we defined this :

class PFSenseRuleModule(PFSenseModuleBase):
   ...
    def __init__(self, module, pfsense=None):
        ....
        self.root_elt = self.pfsense.get_element('filter')

get_element is defined in plugins/module_utils/pfsense.py:

class PFSenseModule(object):
    ...
    def get_element(self, node, root_elt=None, create_node=False):
        """ return <node> configuration element """
        if root_elt is None:
            root_elt = self.root
        elt = root_elt.find(node)
        if elt is None and create_node:
            elt = self.new_element(node)
            root_elt.append(elt)
        return elt

Back to our rule.py. We do a check if tracker is already existent:

class PFSenseModule(object):
    ...
    def _check_tracker(self):
        """ check the tracking used is unique and change it if required """
        if not self.trackers:
            trackers = self.root_elt.findall('tracker')
            for tracker in trackers:
                self.trackers.add(tracker.text)

        start = int(time.time())
        while self.obj['tracker'] in self.trackers:
            start = start + 1
            self.obj['tracker'] = str(start)

        # keep the tracker for future calls if module is used with aggregate
        self.trackers.add(self.obj['tracker'])

thanks to debug print (print(f"self.root_elt: {ET.tostring(self.root_elt, encoding='unicode', method='xml')}"), I can confirm that my ET object self.root_elt is defined and contains my filter node.

Buuuut very surprisingly, tahnks to debug printf print(f"trackers found: {trackers}"), we can confirm that self.root_elt.findall('tracker') returns no values:
trackers found: []

If we check our ET element it has globally this strcuture:

<filter>
		<rule>
			<descr>Debug purposes</descr>
			<interface>wan</interface>
			<type>pass</type>
			<ipprotocol>inet</ipprotocol>
			<statetype>keep state</statetype>
			<source>
				<any />
			</source>
			<destination>
				<any />
			</destination>
			<protocol>icmp</protocol>
			<icmptype>any</icmptype>
			<disabled />
			<id />
			<tracker>1752857769</tracker>
			<created>
				<time>1752857769</time>
				<username>root@<ANSIBLE_CONTROLLER_IP_REDACTED>47184 22</username>
			</created>
			<updated>
				<time>1752857769</time>
				<username>root@<ANSIBLE_CONTROLLER_IP_REDACTED>47184 22</username>
			</updated>
		</rule>
		<rule>
			<descr>Allow openvpn</descr>
			<interface>wan</interface>
			<type>pass</type>
			<ipprotocol>inet</ipprotocol>
			<statetype>keep state</statetype>
			<source>
				<any />
			</source>
			<destination>
				<network>wanip</network>
				<port>1194</port>
			</destination>
			<protocol>tcp/udp</protocol>
			<log />
			<id />
			<tracker>1752857770</tracker>
			<created>
				<time>1752857770</time>
				<username>root@<ANSIBLE_CONTROLLER_IP_REDACTED>47192 22</username>
			</created>
			<updated>
				<time>1752857770</time>
				<username>root@<ANSIBLE_CONTROLLER_IP_REDACTED>47192 22</username>
			</updated>
		</rule>
</filter>

So tracker is not a direct element of my ET object. Then self.root_elt.findall('tracker') fails to find anything.
Instead we should self.root_elt.findall('rule/tracker') or more globally but time consuming XPath instead: self.root_elt.findall('.//tracker') to fix the issue .