-
Notifications
You must be signed in to change notification settings - Fork 247
ASD Conditional Access Policy Evaluation Script
The asd-ca-get.ps1 script evaluates Conditional Access policies in a Microsoft Entra ID (Azure AD) tenant against the Australian Signals Directorate (ASD) Blueprint for Secure Cloud recommendations. It generates a comprehensive HTML report showing compliance status, gaps, detailed remediation guidance, and areas for improvement.
This script helps organizations:
- ✅ Assess compliance with ASD Blueprint security recommendations
- 🔍 Identify gaps in their Conditional Access policy configuration
- 📊 Prioritize remediation efforts based on risk levels
- 📝 Document their security posture for auditing purposes
- 🛠️ Receive specific, actionable remediation steps for every non-compliant policy
- PowerShell 5.1 or later (PowerShell 7+ recommended)
- Microsoft.Graph.Authentication PowerShell module (auto-installed if missing)
- Microsoft Entra ID tenant access with appropriate permissions
The script requires the following Microsoft Graph API permissions:
-
Policy.Read.All- Read Conditional Access policies -
Directory.Read.All- Read tenant and organization information
- Microsoft Entra ID P2 - For Identity Protection risk-based policies
- Microsoft Purview Insider Risk Management - For insider risk conditions
- Authentication Contexts - Must be configured for protected information scenarios
- Authentication Strength - Requires Microsoft Entra ID P1 or P2 for phishing-resistant MFA policies
- Terms of Use - Requires Microsoft Entra ID P1 or P2 for agreement enforcement
- Clone or download the script to your local machine
- Open PowerShell and navigate to the script directory
- Run the script - required modules will be installed automatically if missing
cd C:\Downloads\source\Office365
.\asd-ca-get.ps1NEW (v2.0): The script now handles authentication issues gracefully:
- Automatic Fallback: If interactive browser authentication fails (HTTP listener issues), automatically switches to device code authentication
- Browser Auto-Open: Opens https://microsoft.com/devicelogin automatically when device code is needed
- Clear Instructions: Displays the device code prominently with step-by-step guidance
- Session Detection: Detects existing Microsoft Graph connections and reuses them
The script provides detailed, actionable remediation steps for ALL policies scoring below 100%, including:
Previously, policies scoring 80%+ were marked as "Compliant" with no remediation guidance. Now:
- Shows missing points (e.g., "missing 13/100 points")
- Names specific policy to open in Entra portal
- Provides systematic checklist of configuration areas to verify
- Includes specific values to check (e.g., "4 hours for admins, 8-12 hours for users")
The script analyzes findings and provides specific guidance for:
- Partial sign-in risk configuration → "Update to include all required risk levels (Medium and High)"
- Policy not enabled → "Enable the policy after testing in report-only mode"
- Excessive exclusions → "Review user exclusions, only exclude break-glass accounts"
- Limited app scope → "Verify target resources include all required apps"
- Session frequency issues → "Verify duration: 4 hours for admins, 8-12 for users"
- Platform restrictions → "Review device platforms for full coverage"
For partial and non-compliant policies, remediation includes:
- Exact navigation paths (e.g., "Security > Conditional Access > Policies")
- Specific policy name to open
- Field-level guidance (which checkboxes to select, which dropdowns to configure)
- Recommended values (authentication strengths, session durations, risk levels)
When grant controls don't match recommendations, the report shows:
✗ Grant controls don't match recommendation
Expected: Block access
Current: Require MFA
This applies to all grant control types:
- Block access
- Require MFA / Authentication strength (Phishing-resistant MFA)
- Require compliant device
- Require Hybrid Azure AD joined device
- Require approved client app
- Require app protection policy
- Grant access (with session controls)
NEW (v2.0): Comprehensive validation includes:
- Policy State Validation: Non-enabled policies (disabled/report-only) have scores reduced by 50%
- Exclusion Validation: Checks for significant exclusions that negate policy intent (users, groups, roles, applications)
- Grant Controls Operator Validation: Validates AND vs OR operators when multiple controls are configured
- Authentication Strength Validation: Verifies authentication strength policies use phishing-resistant methods (Windows Hello, FIDO2, x509 certificates)
- Risk Level Graduated Scoring: Explicit validation of high/medium/low risk levels with percentage-based scoring
- Feature Availability Tracking: Detects and reports when premium features are unavailable due to licensing
NEW (v2.0): All criteria are now weighted equally (100 points each) ensuring fair evaluation:
- Eliminates arbitrary weight assignments (previously 20/30/40 points)
- Removes double-counting from bonus points
- Provides consistent maximum scores across all recommendation types
- Makes compliance percentages directly comparable
NEW (v2.0): Thresholds are now documented and configurable:
$complianceThreshold = 80 # Minimum score for "Compliant" status
$partialThreshold = 50 # Minimum score for "Partial" statusRationale Documented:
- 80% threshold allows reasonable implementation flexibility while maintaining security
- Avoids false negatives from legitimate exclusions or phased rollouts
- Organizations can adjust based on risk tolerance (lines 1374-1375)
- ASD Blueprint does not prescribe specific compliance percentages
The report uses color-coded boxes to clearly distinguish information types:
- Blue boxes - Best Matching Policy details (existing policy information)
- Teal/Cyan boxes - Remediation Steps (actionable guidance to achieve 100%)
This ensures remediation guidance stands out without implying false success.
- Checks for Microsoft.Graph.Authentication module
- Installs missing modules automatically
- Displays installation progress with color-coded status
NEW: Enhanced authentication with automatic fallback:
- Checks for existing Microsoft Graph connection
- Attempts interactive browser authentication
- On HTTP listener failure, automatically switches to device code authentication
- Opens browser to https://microsoft.com/devicelogin
- Displays device code prominently in console
- Provides clear troubleshooting guidance if both methods fail
- Retrieves tenant display name
- Gets tenant ID
- Obtains primary verified domain
- Handles errors gracefully with fallback values
- Fetches all Conditional Access policies via Graph API
- Uses direct REST API calls (Invoke-MgGraphRequest)
- Displays count of policies found
Enhanced: Now tracks four premium features:
- Authentication Contexts: Queries configured authentication context class references
- Insider Risk Management: Checks if policies use insider risk conditions
- Authentication Strength: Fetches authentication strength policies and validates phishing-resistant methods
- Terms of Use: Retrieves Terms of Use agreements for validation
Defines 17 recommendations across 5 categories with detailed check criteria for evaluation.
NEW: Enhanced validation functions:
-
Test-PolicyCompliance: Main compliance evaluation with normalized 100-point scoring per criterion -
Test-HasSignificantExclusions: Validates exclusions don't negate policy intent -
Test-GrantControlsOperator: Checks AND vs OR operators for multiple controls -
Test-IsPhishingResistantAuthStrength: Validates authentication methods against phishing-resistant list
For each of the 17 ASD recommendations:
Step 1: Condition-Based Matching
- Analyzes each policy's actual configuration
- Assigns standardized scores (100 points per matched condition)
- Identifies which policies are relevant to each recommendation
Step 2: Match Threshold
- Only policies with ≥50% condition match are considered
- Eliminates false matches from unrelated policies
Step 3: Compliance Scoring
- Evaluates matched policies against recommendation criteria
- Checks: Users, Applications, Grant Controls, Client Apps, Risk Levels, Locations, Sessions
- Calculates compliance percentage (0-100%)
- NEW: Validates policy state, exclusions, operators, authentication strength details
- NEW: Provides detailed "Expected vs Current" comparisons for grant control mismatches
Step 4: Combined Scoring
- Combines condition match (40%) + compliance score (60%)
- Selects best matching policy for each recommendation
Step 5: Status Classification
- Compliant: ≥80% compliance (configurable)
- Partial: 50-79% compliance
- Non-Compliant: <50% compliance
- Missing: No matching policy found
┌─────────────────────────────────────────┐
│ Header │
│ - Title and ASD Blueprint reference │
├─────────────────────────────────────────┤
│ Tenant Information │
│ - Name, Domain, ID, Date, Policy Count │
│ - Feature Availability Status (4 items) │
├─────────────────────────────────────────┤
│ Executive Summary │
│ - Overall Compliance Percentage │
│ - Status Breakdown (Cards) │
│ - Priority Issues (Critical/High/Med/Low)│
├─────────────────────────────────────────┤
│ Detailed Evaluation Results │
│ - Grouped by Category (Collapsible) │
│ ├─ User Protection (10 policies) │
│ ├─ Device Protection (3 policies) │
│ ├─ Admin Protection (1 policy) │
│ ├─ Location Protection (1 policy) │
│ └─ Guest Protection (2 policies) │
│ │
│ Each recommendation shows: │
│ - Priority badge (Critical/High/Med/Low) │
│ - Compliance status badge │
│ - Compliance score & progress bar │
│ - Detailed findings list │
│ - Expected vs Current comparisons 🆕 │
│ - Best matching policy details (Blue) │
│ - Remediation steps box (Teal/Cyan) 🆕 │
├─────────────────────────────────────────┤
│ Important Notes │
│ - Feature Requirements │
│ - Evaluation Methodology │
│ - Threshold Rationale 🆕 │
├─────────────────────────────────────────┤
│ Footer │
│ - ASD Blueprint reference link │
│ - Generation timestamp │
└─────────────────────────────────────────┘
- Collapsible sections - Click category headers and recommendations to expand/collapse
- Expand/Collapse All buttons - Control all sections at once
- Color-coded status badges - Visual indicators for compliance levels
- Progress bars - Visual representation of compliance scores
- Teal/Cyan remediation boxes 🆕 - Clearly distinguished actionable guidance
- Responsive design - Adapts to different screen sizes
- Print-friendly - Optimized CSS for printing
- Saves HTML report to parent directory
- Filename format:
ASD-CA-Evaluation-Report-YYYYMMDD-HHMMSS.html - Opens report in default browser automatically
- Displays summary statistics in console
- Disconnects from Microsoft Graph
- Handles disconnect errors silently
The script evaluates 17 recommendations across 5 categories:
| Recommendation | Description | Priority |
|---|---|---|
| USR - B - Block access via legacy auth | Legacy protocol blocking | Critical |
| USR - G - Require strong auth | Phishing-resistant MFA for all | Critical |
| USR - B - Block high-risk sign-ins | Risk-based sign-in blocking | Critical |
| USR - B - Block high-risk users | Risk-based user blocking | High |
| USR - B - Block users with elevated insider risk | Insider risk management | High |
| USR - G - Register security info with strong auth | MFA for security info registration | High |
| USR - G - Risky sign-ins with strong auth | MFA for risky sign-ins | High |
| USR - B - Block access to PROTECTED information | Authentication context enforcement | Medium |
| USR - S - Limit user sessions | Session duration limits | Medium |
| USR - G - Agreement to terms of use | Terms of use requirement | Low |
| Recommendation | Description | Priority |
|---|---|---|
| DEV - G - Compliant devices | Requires compliant devices | High |
| DEV - B - Block access from unapproved devices | Blocks unapproved device access | High |
| DEV - G - Intune enrolment with strong auth | Strong auth for Intune enrollment | Medium |
| Recommendation | Description | Priority |
|---|---|---|
| ADM - S - Limit admin sessions | Limits administrative session durations | High |
| Recommendation | Description | Priority |
|---|---|---|
| LOC - B - Block access from unapproved countries | Geo-blocking for unauthorized locations | High |
| Recommendation | Description | Priority |
|---|---|---|
| GST - B - Block guests | Blocks guest access where not needed | Medium |
| GST - G - Guest application access with strong auth | Strong auth for guest access | Medium |
Recommendations are classified by priority:
| Priority | Risk Level | Remediation Timeline |
|---|---|---|
| Critical | Highest risk - immediate exploitation possible | Within 24-48 hours |
| High | Significant risk - should be addressed urgently | Within 1-2 weeks |
| Medium | Moderate risk - standard remediation timeline | Within 30 days |
| Low | Lower risk - can be addressed in planned updates | Within 90 days |
Policies are matched to recommendations based on their configuration. Each matched condition receives 100 points.
Each criterion type receives equal weight (100 points) when evaluated:
- Users assignment
- Applications scope
- Grant controls
- Client apps
- Sign-in risk levels
- User risk levels
- Locations
- Session controls
- User actions
- Device platforms
Final Score = (Condition Match × 0.4) + (Compliance Score × 0.6)
This ensures policies are both relevant (condition match) and properly configured (compliance).
┌──────────────┬────────────┬───────────────────────────────┐
│ Status │ Score │ Interpretation │
├──────────────┼────────────┼───────────────────────────────┤
│ Compliant │ ≥80% │ Meets ASD recommendations │
│ Partial │ 50-79% │ Partially configured │
│ Non-Compliant│ <50% │ Significant gaps exist │
│ Missing │ 0% │ No matching policy found │
└──────────────┴────────────┴───────────────────────────────┘
Threshold Rationale:
- 80% Threshold: Allows for minor gaps (legitimate exclusions, conditional variations) while ensuring core security requirements are met. Reduces false negatives in real-world deployments.
- 50% Threshold: Distinguishes between policies that are "close" (can be remediated) vs fundamentally inadequate (require rebuild).
- Note: The ASD Blueprint does not prescribe specific compliance percentages. Organizations may adjust these values (lines 1374-1375) based on risk tolerance. Some frameworks use 90%, but this can flag legitimate implementations as non-compliant.
NEW: Even "Compliant" policies (80-99%) now receive remediation guidance to achieve 100% alignment.
Reports are saved to the parent directory of the script location:
Script: C:\Downloads\source\Office365\asd-ca-get.ps1
Report: C:\Downloads\source\ASD-CA-Evaluation-Report-YYYYMMDD-HHMMSS.html
ASD-CA-Evaluation-Report-[YYYYMMDD]-[HHMMSS].html
Example: ASD-CA-Evaluation-Report-20251126-143022.html
Checking for required PowerShell modules...
✓ Microsoft.Graph.Authentication found
All required modules are installed
Connecting to Microsoft Graph...
Successfully connected to Microsoft Graph
Retrieving tenant information...
Tenant: Contoso Ltd
Evaluating Conditional Access policies for tenant: Contoso Ltd
Retrieving Conditional Access policies...
Found 15 Conditional Access policies
Checking for authentication contexts...
Found 2 authentication context(s)
Checking for Insider Risk Management features...
No policies with Insider Risk conditions found
Note: Insider Risk Management requires Microsoft Purview and additional licensing
Retrieving authentication strength policies...
Found 5 authentication strength policy/policies
Checking for Terms of Use...
Found 1 Terms of Use agreement(s)
Evaluating: ADM - S - Limit admin sessions
Status: Compliant (85%)
Best matching policy: CA001-AdminSessionLimits
[... additional evaluations ...]
============================================
Evaluation Complete!
============================================
Report saved to:
C:\Downloads\source\ASD-CA-Evaluation-Report-20251126-143022.html
Overall Compliance: 72%
Summary:
✓ Compliant: 8 / 17
⚠ Partial: 5 / 17
✗ Non-Compliant: 2 / 17
⊘ Missing: 2 / 17
Report opened in default browser
Disconnected from Microsoft Graph
Error: Failed to install Microsoft.Graph.Authentication
Solution:
# Install manually with elevated permissions
Install-Module -Name Microsoft.Graph.Authentication -Scope CurrentUser -ForceError: InteractiveBrowserCredential authentication failed: HttpListenerException
Solution (Automatic): The script now automatically switches to device code authentication when this error occurs.
Solution (Manual Fix):
# Run as Administrator
netsh http add iplisten 127.0.0.1Error: Failed to retrieve tenant information
Result: Script continues with "Unknown" values
Impact: Report generated but tenant details not shown
Error: Failed to retrieve Conditional Access policies
Result: Script exits
Solution: Verify Policy.Read.All permission is granted
- Review permissions - Ensure account has necessary Graph API permissions
- Check licensing - Verify tenant has required licenses for advanced features
- Plan timing - Run during maintenance windows for large tenants
- Backup existing policies - Export current policies before making changes
- Review report thoroughly - Examine all findings and priorities
- Address Critical issues first - Focus on highest risk gaps
- Follow remediation steps 🆕 - Use the specific portal instructions provided
- Test changes - Use report-only mode or test policies before enforcement
- Document decisions - Note why certain recommendations may not apply
- Schedule regular reviews - Re-run quarterly or after significant changes
-
Enable verbose output - Run with
-Verboseparameter (if added) - Check Graph API status - Verify Microsoft 365 service health
- Review permissions - Confirm consent was granted for all required scopes
-
Test connectivity - Use
Test-MgGraphConnectionor similar cmdlets
For automation or scheduled tasks, pre-connect with device code authentication:
# Connect with device code authentication manually
Connect-MgGraph -Scopes 'Policy.Read.All','Directory.Read.All' -UseDeviceAuthentication
# Run script (will detect existing connection)
.\asd-ca-get.ps1For certificate-based authentication in automation:
# Connect with certificate-based authentication
Connect-MgGraph -ClientId $AppId -TenantId $TenantId -CertificateThumbprint $Thumbprint
# Run script
.\asd-ca-get.ps1The script can be modified to focus on specific categories or priorities:
# Filter recommendations before evaluation
$asdRecommendations = $asdRecommendations | Where-Object { $_.Priority -eq "Critical" }Modify lines around 1403 to change output directory:
# Default (parent directory)
$reportPath = Join-Path (Split-Path $PSScriptRoot -Parent) "ASD-CA-Evaluation-Report-$reportDate.html"
# Custom location
$reportPath = "C:\Reports\ASD-CA-Evaluation-Report-$reportDate.html"Modify lines 1374-1375 to adjust thresholds based on organizational requirements:
# Default thresholds
$complianceThreshold = 80 # Minimum score for "Compliant" status
$partialThreshold = 50 # Minimum score for "Partial" status
# Stricter compliance (may increase false negatives)
$complianceThreshold = 90
$partialThreshold = 60
# More lenient assessment
$complianceThreshold = 70
$partialThreshold = 40- Name-independent matching - Does not rely on policy naming conventions
- Feature detection - Best-effort detection of advanced features (Insider Risk, Auth Contexts)
- Single tenant - Evaluates one tenant per execution
- Read-only - Does not modify existing policies
- Manual remediation - Does not auto-create missing policies (provides detailed guidance instead) 🆕
- Licensing awareness - Some recommendations require specific licenses
- Uses read-only Graph API permissions
- Does not modify or create policies
- Interactive authentication preferred for security
- Certificate-based auth recommended for automation
- Generates local HTML report only
- No data sent to external services
- Report contains sensitive tenant information
- Store reports securely with appropriate access controls
- Does not store credentials
- Relies on Microsoft Graph SDK authentication
- Session tokens managed by PowerShell SDK
- Disconnects after execution
This script evaluates policies against the ASD Blueprint for Secure Cloud recommendations as of November 2025.
While designed for ASD Blueprint, findings may align with:
- Essential Eight - Maturity Level 2 & 3 requirements
- ISO 27001 - Access control standards
- NIST CSF - Identity and access management
- CIS Controls - Account management controls
- Report timestamp shows evaluation date/time
- Tenant information identifies evaluated environment
- Compliance scores provide historical tracking
- Can be used for audit evidence
- ASD Blueprint: https://blueprint.asd.gov.au/
- Microsoft Graph API: https://learn.microsoft.com/graph/
- Conditional Access: https://learn.microsoft.com/entra/identity/conditional-access/
- Authentication Strength: https://learn.microsoft.com/entra/identity/authentication/concept-authentication-strengths
- Review quarterly for ASD Blueprint updates
- Check for Graph API changes or deprecations
- Update recommendation weights based on threat landscape
- Test with new Microsoft Entra ID features
For issues or improvements:
- Document the issue or enhancement
- Test proposed changes thoroughly
- Update this documentation accordingly
- Consider backward compatibility
Major Enhancements:
- ✨ Enhanced Authentication: Automatic fallback to device code authentication with browser auto-open
- 🔒 Advanced Validation: Policy state, exclusions, operators, authentication strength, risk levels
- 📊 Normalized Scoring: Equal weight (100 points) for all criteria types
- 📏 Configurable Thresholds: Documented and adjustable compliance thresholds (80%/50%)
- 🔐 Phishing-Resistant MFA: Validates authentication strength against phishing-resistant methods
- 🎯 Feature Availability: Tracks authentication strength and Terms of Use availability
- ✅ Grant Access Validation: Proper validation of "Grant access" control type
- 🎭 Guest Policy Differentiation: Separates block-guests from guest-app-access requirements
- 📈 Combined Scoring Consistency: Uses combined score for both selection and status
- 🔧 Syntax Fix: Resolved extra closing brace causing parser error (line 1279)
Technical Improvements:
- Enhanced authentication with device code fallback (Lines 86-123)
- Added feature availability tracking for all premium features (Lines 159-230)
- Normalized scoring to 100 points per criterion (Lines 430-1088)
- Added helper functions for validation (Test-HasSignificantExclusions, Test-GrantControlsOperator, Test-IsPhishingResistantAuthStrength)
- Fixed combined scoring usage consistency (Lines 1360-1380)
- Added configurable compliance thresholds with documentation (Lines 1364-1379)
- Enhanced HTML report with feature availability status (Lines 1913-1926)
- Updated prerequisites documentation (Lines 2448-2453)
Major Enhancements:
- ✨ Comprehensive Remediation Engine: ALL policies <100% now receive specific remediation guidance
- 📊 Detailed Grant Control Mismatches: Shows "Expected vs Current" comparisons for grant controls
- 🎯 Intelligent Issue Detection: Analyzes findings to provide targeted remediation
- 🗺️ Portal Navigation Steps: Exact paths and field-level guidance for Entra portal
- 🎨 Visual Differentiation: Teal/Cyan remediation boxes distinct from blue policy info boxes
- 💡 High-Score Remediation: Policies scoring 80-99% get specific guidance (previously only <80%)
- 🔍 Missing Points Display: Shows exact points missing (e.g., "missing 13/100 points")
- ✅ Systematic Checklists: Comprehensive verification steps for near-compliant policies
Technical Improvements:
- Enhanced remediation suggestion generation (Lines 1554-1878)
- Added grant control translation for human-readable output
- Improved HTML report styling with distinct color schemes
- Added fallback remediation logic for edge cases
- Initial release
- 17 ASD Blueprint recommendations
- Condition-based policy matching
- Interactive HTML report generation
- Feature availability detection
- Comprehensive error handling
- ASD Blueprint for Secure Cloud
- Conditional Access Documentation
- Microsoft Graph API - Conditional Access
- Authentication Strength
- Identity Protection
- Insider Risk Management
Author: CIAOPS
Version: 2.0
Last Updated: 2025-11-26
License: Review repository for license details
Script Location: GitHub - directorcia/Office365