🛡️ Sentinel: [HIGH] Fix XXE vulnerabilities in XML parsing

.jules/sentinel.md

@@ -0,0 +1,4 @@
+## 2024-05-20 - Prevent XML External Entity (XXE) and Billion Laughs Vulnerabilities
+**Vulnerability:** Use of standard library `xml.etree.ElementTree` to parse untrusted XML/RSS feeds in `theverge.py` and `producthunt.py` scrapers.
+**Learning:** The built-in `xml.etree` module in Python is vulnerable to malicious XML payloads such as XML External Entities (XXE) and Billion Laughs attacks. Parsing feeds from external, untrusted sources without defensive measures creates severe security risks (DoS or data exfiltration).
+**Prevention:** Always use `defusedxml.ElementTree` instead of `xml.etree.ElementTree` when parsing any XML data from an untrusted source or network request. Ensure `defusedxml` is included in project dependencies.

functions/requirements.txt

@@ -7,3 +7,4 @@ beautifulsoup4==4.*
 feedparser==6.*
 openai==1.*
 tzdata
+defusedxml==0.*

functions/scrapers/producthunt.py

@@ -6,7 +6,7 @@
 import httpx
 from typing import List, Dict, Any
 from datetime import datetime
-import xml.etree.ElementTree as ET
+import defusedxml.ElementTree as ET
 from bs4 import BeautifulSoup
 
 
@@ -114,8 +114,12 @@ def fetch_producthunt(limit: int = 10) -> List[Dict[str, Any]]:
                 for entry in entries[:limit]:
                     title = entry.find('atom:title', atom_ns)
                     link = entry.find('atom:link', atom_ns)
-                    summary = entry.find('atom:summary', atom_ns) or entry.find('atom:content', atom_ns)
-                    published = entry.find('atom:published', atom_ns) or entry.find('atom:updated', atom_ns)
+                    summary = entry.find('atom:summary', atom_ns)
+                    if summary is None:
+                        summary = entry.find('atom:content', atom_ns)
+                    published = entry.find('atom:published', atom_ns)
+                    if published is None:
+                        published = entry.find('atom:updated', atom_ns)
 
                     if title is None or link is None:
                         continue

functions/scrapers/theverge.py

@@ -6,7 +6,7 @@
 import httpx
 from typing import List, Dict, Any
 from datetime import datetime
-import xml.etree.ElementTree as ET
+import defusedxml.ElementTree as ET
 
 try:
     from ..resilience import retry_with_backoff