Skip to content

ASD Organization Services Configuration Check

Jump to bottom
directorcia edited this page Nov 13, 2025 · 1 revision

asd-orgservices-get.ps1 Documentation

Purpose: Assess Microsoft 365 (M365) Organization / Tenant service configuration against an ASD / baseline JSON, producing console + HTML reports and distinguishing API‑verifiable vs GUI‑only settings (marked MANUAL_CHECK_REQUIRED).

1. Overview

asd-orgservices-get.ps1 loads a baseline definition (local file or GitHub URL) of required values for tenant services, retrieves what it can via Microsoft Graph REST (device code auth), compares results, and outputs:

  • Colored console summary
  • HTML compliance report (cards, table, manual verification badge)
  • CSV/JSON (if extended output is later added – placeholders for future enhancement)

The script intentionally avoids the Microsoft Graph PowerShell module for reliability, using raw REST calls via an acquired OAuth access token.

2. Key Features

  • Device code authentication (clipboard & browser launch convenience)
  • Minimal required delegated Graph scopes (recommended start: openid profile offline_access Organization.Read.All Reports.Read.All)
  • Separation of verifiable vs manual-only checks to prevent false failure reporting
  • Extensible baseline-driven architecture (add/remove services or settings without code change)
  • HTML dashboard with compliance percentage excluding manual (GUI-only) checks

3. Current Service Coverage

The retrieval layer (Get-ServiceSettings) currently implements partial / placeholder coverage:

Service Name API Retrieval Status Notes
ModernAuthentication Placeholder values only True/False placeholders until org settings endpoint confirmed
Reports Reads admin/reportSettings (display concealment) Requires Reports.Read.All; may expand
Others (baseline-defined but not coded) Manual only Marked MANUAL_CHECK_REQUIRED (no endpoint implemented yet)

Any baseline service without a coded branch or whose endpoint returns no data is marked MANUAL_CHECK_REQUIRED.

4. When a Setting Becomes Manual

A check is flagged MANUAL_CHECK_REQUIRED if either:

  • There is no implemented Graph endpoint for the service/setting
  • The endpoint does not return a value (null) / is inaccessible
  • The setting exists only in a GUI blade (security/compliance portals, admin centers) with no public Graph surface

These manual items are excluded from the compliance percentage. The HTML report status logic:

  • All checks manual → Overall: MANUAL VERIFICATION REQUIRED
  • No failed verifiable checks + some manual → COMPLIANT (API) - Manual Checks Pending
  • Some failures among verifiable → NON-COMPLIANT
  • All verifiable pass and no manual → COMPLIANT

5. File & Function Landmarks

Function Purpose
Get-GraphAccessToken Device code auth – obtains and caches token
Invoke-MSGraphAPI Generic REST GET wrapper with error handling
Test-Setting Compares required vs current or sets manual status
Get-ServiceSettings Service-specific retrieval (switch statement)
Invoke-OrganizationServicesCheck Iterates baseline, calls compare logic
New-HTMLReport Builds styled compliance dashboard

Baseline loading & schema validation: Get-BaselineSettings + Test-BaselineSchema.

6. Prerequisites

  • PowerShell 7+ recommended (Core); script may run on Windows PowerShell but not guaranteed
  • Internet access to authenticate and optionally load baseline from GitHub
  • Ability for signed-in user to consent scopes or existing admin pre-consent
  • Outbound HTTPS to login.microsoftonline.com and graph.microsoft.com

7. Authentication & Scopes

Recommended delegated scopes (minimum for current implementation):

openid profile offline_access Organization.Read.All Reports.Read.All

Additional scopes may be required as you implement new service endpoints (e.g. AuditLog.Read.All, SecurityEvents.Read.All, Policy.Read.All). Always add the least privilege necessary.

If encountering repeated admin consent dialogs:

  1. Ask a Global / Privileged Role Administrator to pre-consent scopes via the Entra admin center.
  2. Confirm the app registration (if using one) or device flow context is within the tenant (not cross-account).
  3. Validate the token contains requested scopes (decode at https://jwt.ms).

8. Running the Script

Basic invocation (baseline from default GitHub URL):

pwsh ./asd-orgservices-get.ps1

Specify custom baseline:

pwsh ./asd-orgservices-get.ps1 -BaselinePath "C:\Baselines\services.json"

Enable debug verbose logging (if a flag is exposed – or set $env:DEBUG=1 before running if the script checks it):

$env:DEBUG = 1
pwsh ./asd-orgservices-get.ps1

9. Baseline JSON Structure

Expected shape:

{
  "ModernAuthentication": {
    "EnableModernAuth": true,
    "EnableAuthenticatedSMTP": false
  },
  "Reports": {
    "DisplayConcealedNames": true
  }
  // Additional services...
}

Rules:

  • Top-level properties = service names
  • Each service property object = individual setting:value pairs
  • Values can be bool, string, number, or array (current compare supports simple equality & array membership equality)

10. Extending Service Coverage

  1. Add baseline settings in your JSON.
  2. Implement retrieval logic inside Get-ServiceSettings switch:
"NewService" {
  $raw = Invoke-MSGraphAPI -Uri "security/someEndpoint"  # example
  if ($raw) { $settings["SomeSetting"] = [bool]$raw.propertyFlag }
}
  1. Map raw response values to primitive comparable values (bool/string/int/array)
  2. Re-run script; verifiable checks will now contribute to compliance percentage.

11. Output Interpretation

Console summary example (simplified):

Total Checks     : 25
Verifiable (API) : 8
Passed (API)     : 7
Failed (API)     : 1
Manual (GUI)     : 17
API Compliance   : 87.50%
Overall Status   : COMPLIANT (API) - Manual Checks Pending

HTML report cards:

  • Total Checks: all baseline items
  • Passed API / Failed API: only verifiable items
  • GUI Required: manual checks count
  • API Compliance (%): Passed / Verifiable * 100 Status badge color: Green (compliant), Amber (all manual), Red (non-compliant).

12. Manual GUI Navigation Guide

Below are step-by-step paths for typical settings commonly absent from Graph today. Adapt names to match your baseline.

12.1 Modern Authentication (Exchange Online / Org-wide)

Goal: Confirm modern auth is enforced; legacy protocols disabled where required.

  1. Sign in to Microsoft 365 admin center: https://admin.microsoft.com
  2. Navigate: Settings → Org settings → Modern authentication.
  3. Verify: "Enable modern authentication" is checked (per baseline EnableModernAuth=true).
  4. If baseline requires SMTP Auth disabled, ensure "Authenticated SMTP" toggle is OFF (EnableAuthenticatedSMTP=false).
  5. Save any changes; record observed values manually if script shows MANUAL_CHECK_REQUIRED.

Alternate deeper checks (Exchange-specific legacy protocols):

  1. Open Exchange Admin Center: https://admin.exchange.microsoft.com
  2. Go to Settings → Authentication (preview) or (classic) Organization → Settings → Modern authentication.
  3. Validate legacy protocol toggles (POP3/IMAP, Exchange ActiveSync) align with security baseline.

12.2 Reports (Concealed Names / Privacy)

Goal: Ensure user privacy in usage reports.

  1. Microsoft 365 admin center: https://admin.microsoft.com
  2. Reports → Settings (gear icon) or Reports → Usage → choose any report → top banner "Report settings".
  3. Verify "Display concealed user, group, and site names" matches baseline (often true for privacy).
  4. If mismatch, adjust and save; refresh script after API propagation (may take minutes).

12.3 Azure AD (Entra) Security Defaults / Conditional Access (if baseline includes)

  1. Entra admin center: https://entra.microsoft.com
  2. Navigate: Protection → Conditional Access → Policies.
  3. Manually correlate required policies (naming, enabled state, assignments). Many CA attributes not wholly accessible via simple baseline currently.
  4. For Security Defaults: Identity → Overview → Properties → Manage Security defaults.
  5. Record enabled/disabled status for each baseline policy item.

12.4 Audit / Unified Audit Log (Security & Compliance)

  1. Microsoft Purview compliance portal: https://purview.microsoft.com (or https://compliance.microsoft.com)
  2. Audit → Audit search.
  3. Confirm audit log is enabled (banner indicates state). If baseline expects AuditLogEnabled=true, ensure active; enabling may be tenant-wide already.

12.5 Defender for Office 365 Safe Attachments / Links

  1. Defender portal: https://security.microsoft.com
  2. Policies & rules → Threat policies → Safe Attachments.
  3. Verify policy mode (e.g., Dynamic Delivery) per baseline.
  4. Safe Links: Policies & rules → Safe Links → Global settings & policies.
  5. Note values (e.g., Do not let users click through, Track click) where baseline enumerates.

12.6 SharePoint / OneDrive Sharing Policy

  1. SharePoint admin center: https://admin.microsoft.com → Expand "Admin centers" → SharePoint.
  2. Policies → Sharing.
  3. Confirm external sharing level per baseline (e.g., "Existing guests only" or "New and existing guests").

12.7 Teams External Access & Guest Access

  1. Teams admin center: https://admin.teams.microsoft.com
  2. Users → External access; confirm allowed domains settings.
  3. Users → Guest access; verify toggles (meetings, messaging) meet baseline.

12.8 MFA / Authentication Strength (Conditional Access)

  1. Entra admin center → Protection → Conditional Access → Authentication strengths.
  2. Confirm required authentication strength assigned to key policies.
  3. Baseline may define RequirePhishingResistantMFA=true.

12.9 Legacy Protocol Disabling (Exchange PowerShell Fallback)

Some legacy flags may require Exchange Online PowerShell module:

Install-Module ExchangeOnlineManagement
Connect-ExchangeOnline
Get-OrganizationConfig | Select OAuth2ClientProfileEnabled

If baseline needs OAuth2ClientProfileEnabled true → ensures modern auth. Document results manually.

12.10 General Manual Recording Template

Use a tracking table (CSV or section in baseline) for manual verification notes:

Service Setting Required Observed Verified By Date
ModernAuthentication EnableModernAuth true true admin@example.com 2025-11-13
Reports DisplayConcealedNames true true admin@example.com 2025-11-13

13. Mapping Manual Items to Portals

Baseline Service (example) Typical Portal Navigation Path
ModernAuthentication M365 Admin / Exchange Settings → Org settings → Modern authentication
Reports M365 Admin Reports → Report settings
ConditionalAccessPolicies Entra Protection → Conditional Access → Policies
SecurityDefaults Entra Identity → Properties → Security defaults
SafeLinks Defender Policies & rules → Threat policies → Safe Links
SafeAttachments Defender Policies & rules → Threat policies → Safe Attachments
AuditLog Purview Audit → Audit search (status banner)
SharePointSharing SharePoint Admin Policies → Sharing
TeamsGuestAccess Teams Admin Users → Guest access
TeamsExternalAccess Teams Admin Users → External access

Adjust names to match real baseline keys.

14. Adding Portal Path Metadata (Optional Enhancement)

Consider enriching baseline JSON with helper metadata:

{
  "ModernAuthentication": {
    "EnableModernAuth": { "required": true, "portalPath": "Admin Center > Org settings > Modern authentication" },
    "EnableAuthenticatedSMTP": { "required": false, "portalPath": "Admin Center > Org settings > Modern authentication" }
  }
}

Then extend script to parse objects with { "required": value, "portalPath": "..." } for automatic inclusion in HTML (future enhancement proposal).

15. Troubleshooting

Symptom Possible Cause Action
Repeated consent prompts Scope not admin-consented Admin pre-consent via Entra → Enterprise applications → (App) → Permissions
400 BadRequest on organization Missing or insufficient scope / endpoint variant Add Directory.Read.All or use GET https://graph.microsoft.com/v1.0/organization explicitly
Manual items inflate failure count Using older script version Update to latest (GUI items excluded from compliance)
HTML report shows 0 verifiable All services lack implemented API retrieval Implement endpoints in Get-ServiceSettings
Token lacks scopes Device code used on wrong account Re-run auth ensuring correct tenant context

16. Security Considerations

  • Store baseline file in version control; review changes via pull requests.
  • Limit who can run assessment scripts (tokens expose read scopes).
  • Avoid adding write/delete scopes unless strictly required.
  • Rotate client secrets (if moving from device code to confidential client flow).

17. Change Procedure for New Settings

  1. Add setting in baseline file with required value.
  2. Run script → verify it appears as MANUAL_CHECK_REQUIRED.
  3. Implement retrieval logic + map to boolean/string.
  4. Test with debugging enabled.
  5. Validate HTML compliance now includes setting in API counts.

18. Future Enhancements (Suggested Roadmap)

  • Portal path metadata integration
  • Export CSV / JSON structured output
  • Parallel Graph calls for performance
  • Authentication strengths & conditional access extraction (policy parsing)
  • Support for multiple baselines (e.g., Essential Eight maturity levels)

19. Quick Reference Commands

Acquire updated token (if cached token expired):

pwsh ./asd-orgservices-get.ps1

Decode token to verify scopes:

(Get-GraphAccessToken).AccessToken | Out-File token.txt
# Open https://jwt.ms and paste contents of token.txt

Run with custom baseline:

pwsh ./asd-orgservices-get.ps1 -BaselinePath "https://raw.githubusercontent.com/yourorg/security-baselines/main/services.json"

20. Glossary

Term Definition
Verifiable Check Baseline item retrieved & compared via API
Manual Check Baseline item requiring GUI navigation or unsupported endpoint
Compliance % (Passed Verifiable / Total Verifiable) * 100
Baseline JSON file defining required configuration values

21. Disclaimer

This framework does not guarantee complete coverage of all tenant security controls. Manual review remains essential for GUI-only settings until equivalent Graph surfaces are published or implemented.

Last Updated: 2025-11-13

Clone this wiki locally