Add automatic dns-01/http-01 challenge detection for Let's Encrypt via Azure DNS REST API

README.md

@@ -2,7 +2,28 @@ Forked and adapted from https://github.com/morpht/letsencrypt_drupal
 
 # Let's Encrypt Drupal
 
-Wrapper script for https://github.com/dehydrated-io/dehydrated opinionated towards running in Drupal hosting environments and reporting to Slack. Slack is optional. Let's Encrypt challenge is published trough Drupal using Drush. There is no need to alter webserver settings or upload files.
+Wrapper script for https://github.com/dehydrated-io/dehydrated opinionated towards running in Drupal hosting environments and reporting to Slack. Slack is optional. 
+
+## Challenge Types
+
+This script automatically selects the appropriate challenge type based on the domains in your configuration:
+
+### http-01 (Automatic for Subdomains)
+Used automatically for subdomain certificates (e.g., `www.example.com`, `api.example.com`).
+* Let's Encrypt challenge is published through Drupal using Drush
+* Requires: https://www.drupal.org/project/letsencrypt_challenge module on target site
+* No web server configuration changes needed
+
+### dns-01 (Automatic for Apex/Wildcard Domains)
+Used automatically when your domains file contains:
+* **Apex domains**: `example.com`
+* **Wildcard domains**: `*.example.com`
+
+Let's Encrypt challenge is published via DNS TXT records on Azure DNS using REST API.
+* Requires: Azure DNS zone and Service Principal with DNS Zone Contributor permissions
+* Benefits: Supports wildcard certificates, no web server access needed, works behind firewalls
+
+**Challenge type is determined automatically** - the script analyzes your domains file and selects the appropriate validation method.
 
 ## What it does
 
@@ -31,12 +52,20 @@ Wrapper script for https://github.com/dehydrated-io/dehydrated opinionated towar
 
 ## Requirements
 
+### For http-01 challenge (default):
 * Environment where you can run bash script and setup cron.
 * Read access to project root. (accessing config files)
 * Permissions to run Drush commands with Drush alias against the site which is accessible via domains listed in `domains_site.env.txt` from internet.
 * `git` must available.
 * https://www.drupal.org/project/letsencrypt_challenge on target site.
 
+### Additional requirements for dns-01 challenge:
+* Azure DNS zone hosting your domain
+* Azure Service Principal with the following:
+  * Subscription ID, Tenant ID, Client ID, Client Secret
+  * DNS Zone Contributor role on the DNS Zone or Resource Group
+* Environment variables configured (see Azure DNS Configuration section below)
+
 ## Installation
 
 These steps are for `prod` environment of PROJECT on Acquia Cloud. Can be easily adapted to other hosting environments.
@@ -74,7 +103,7 @@ These steps are for `prod` environment of PROJECT on Acquia Cloud. Can be easily
     * `secrets.settings.php`
       * Should *not* be committed in project repository.
       * Should be placed on Acquia server here: `/mnt/files/undp.01live/secrets.settings.php`
-  * Add https://www.drupal.org/project/letsencrypt_challenge module.
+  * Add https://www.drupal.org/project/letsencrypt_challenge module (for http-01 challenge, used with subdomain certificates).
     * `composer require drupal/letsencrypt_challenge`
   * Commit and deploy to production.
 * In Acquia UI add the Scheduled task
@@ -85,6 +114,60 @@ These steps are for `prod` environment of PROJECT on Acquia Cloud. Can be easily
   * New job:
     * Job name: `LE renew cert` (just a default, feel free change it)
     * Command: `/home/undp/letsencrypt_drupal/letsencrypt_drupal.sh undp 01live &>> /var/log/sites/${AH_SITE_NAME}/logs/$(hostname -s)/letsencrypt_drupal.log`
+    * Note: Challenge type (http-01 or dns-01) is automatically detected based on your domains file
     * Command frequency `0 7 * * 1` ( https://crontab.guru/#0_7_*_*_1 )
   * It's good idea to run the command on Acquia manually first time to check if all is OK.
 * First script run will post results/instructions to Slack/Teams.
+
+## Azure DNS Configuration (for dns-01 challenge)
+
+The script automatically uses dns-01 challenge when it detects apex domains (`example.com`) or wildcard domains (`*.example.com`) in your domains file. To enable this functionality, you need to configure Azure DNS:
+
+### 1. Create Azure Service Principal
+
+```bash
+# Login to Azure
+az login
+
+# Create a Service Principal
+az ad sp create-for-rbac --name "letsencrypt-dns-challenge" --role "DNS Zone Contributor" --scopes /subscriptions/{subscription-id}/resourceGroups/{resource-group}/providers/Microsoft.Network/dnszones/{zone-name}
+```
+
+This command will output:
+- `appId` (use as AZURE_CLIENT_ID)
+- `password` (use as AZURE_CLIENT_SECRET)
+- `tenant` (use as AZURE_TENANT_ID)
+
+### 2. Configure Environment Variables
+
+Add these to your config file (e.g., `config_undp.01live.sh`) or store them securely in a secrets file:
+
+```bash
+export AZURE_SUBSCRIPTION_ID="your-subscription-id"
+export AZURE_TENANT_ID="your-tenant-id"
+export AZURE_CLIENT_ID="your-client-id"
+export AZURE_CLIENT_SECRET="your-client-secret"
+export AZURE_RESOURCE_GROUP="your-resource-group"
+export AZURE_DNS_ZONE="example.com"
+```
+
+**Security Note**: Never commit these credentials to your repository. Store them in:
+- `/mnt/files/PROJECT.ENV/secrets.settings.php` on Acquia Cloud
+- Or use environment variables set in your hosting platform
+- Or source from a separate secrets file that is not version controlled
+
+### 3. Verify Permissions
+
+Ensure the Service Principal has "DNS Zone Contributor" role on your DNS Zone:
+
+```bash
+az role assignment list --assignee {client-id} --scope /subscriptions/{subscription-id}/resourceGroups/{resource-group}/providers/Microsoft.Network/dnszones/{zone-name}
+```
+
+### 4. Test the Configuration
+
+Run the script manually first to verify everything works:
+
+```bash
+./letsencrypt_drupal.sh projectname environment dns-01
+```

TESTING.md

@@ -0,0 +1,228 @@
+# Testing Guide for Automatic Challenge Type Selection
+
+This guide explains how to test the automatic challenge type detection in letsencrypt_drupal.
+
+## How Challenge Type is Determined
+
+The script automatically selects the challenge type based on your domains file:
+
+- **dns-01 challenge**: Used when the domains file contains:
+  - Apex domains (e.g., `example.com`)
+  - Wildcard domains (e.g., `*.example.com`)
+
+- **http-01 challenge**: Used for all other cases:
+  - Subdomains only (e.g., `www.example.com`, `api.example.com`)
+
+## Prerequisites for dns-01 Testing
+
+Before testing with apex or wildcard domains, ensure you have:
+
+1. **Azure DNS Zone** configured with your domain
+2. **Azure Service Principal** with DNS Zone Contributor permissions
+3. **Required environment variables** set:
+   - `AZURE_SUBSCRIPTION_ID`
+   - `AZURE_TENANT_ID`
+   - `AZURE_CLIENT_ID`
+   - `AZURE_CLIENT_SECRET`
+   - `AZURE_RESOURCE_GROUP`
+   - `AZURE_DNS_ZONE`
+
+## Testing Automatic Detection
+
+### Test Case 1: Subdomains Only (http-01)
+
+Create a domains file with only subdomains:
+```
+www.example.com api.example.com test.example.com
+```
+
+Run the script:
+```bash
+./letsencrypt_drupal.sh projectname environment
+```
+
+Expected behavior:
+- Automatically detects subdomains
+- Uses `hooks/letsencrypt_drupal_hooks.sh`
+- Challenge type: `http-01`
+- Publishes challenges via Drupal module
+
+### Test Case 2: Apex Domain (dns-01)
+
+Create a domains file with an apex domain:
+```
+example.com www.example.com
+```
+
+Run the script:
+```bash
+./letsencrypt_drupal.sh projectname environment
+```
+
+Expected behavior:
+- Automatically detects apex domain `example.com`
+- Uses `hooks/azure_dns_hook.sh`
+- Challenge type: `dns-01`
+- Creates TXT records in Azure DNS
+- Waits 60 seconds for DNS propagation
+- Cleans up TXT records after validation
+
+### Test Case 3: Wildcard Domain (dns-01)
+
+Create a domains file with a wildcard domain:
+```
+*.example.com www.example.com
+```
+
+Run the script:
+```bash
+./letsencrypt_drupal.sh projectname environment
+```
+
+Expected behavior:
+- Automatically detects wildcard domain `*.example.com`
+- Uses `hooks/azure_dns_hook.sh`
+- Challenge type: `dns-01`
+- Creates TXT records in Azure DNS
+
+## Manual Verification Steps
+
+### 1. Check Log Output
+
+Look for domain detection messages:
+
+```
+Detected apex domain: example.com - using dns-01 challenge
+Using challenge type: dns-01
+Using Azure DNS hook for dns-01 challenge
+```
+
+or for wildcard:
+
+```
+Detected wildcard domain: *.example.com - using dns-01 challenge
+Using challenge type: dns-01
+Using Azure DNS hook for dns-01 challenge
+```
+
+or for subdomains only:
+
+```
+Using challenge type: http-01
+Using Drupal hook for http-01 challenge
+```
+
+### 2. Verify Hook Script Selection
+
+Check that the correct hook is being used:
+
+```bash
+# For dns-01, should show azure_dns_hook.sh
+grep 'HOOK=' /tmp/letsencrypt_drupal/baseconfig
+```
+
+### 3. Test Azure API Authentication
+
+Create a simple test script to verify Azure credentials work:
+
+```bash
+#!/usr/bin/env bash
+
+# Source your config file with Azure credentials
+source /path/to/config_project.env.sh
+
+# Test getting an access token
+TOKEN_ENDPOINT="https://login.microsoftonline.com/${AZURE_TENANT_ID}/oauth2/v2.0/token"
+
+RESPONSE=$(curl -s -X POST "$TOKEN_ENDPOINT" \
+    -H "Content-Type: application/x-www-form-urlencoded" \
+    -d "client_id=${AZURE_CLIENT_ID}" \
+    -d "client_secret=${AZURE_CLIENT_SECRET}" \
+    -d "scope=https://management.azure.com/.default" \
+    -d "grant_type=client_credentials")
+
+if echo "$RESPONSE" | grep -q "access_token"; then
+    echo "✓ Azure authentication successful"
+else
+    echo "✗ Azure authentication failed"
+    echo "$RESPONSE"
+fi
+```
+
+### 4. Verify DNS Record Creation
+
+During dns-01 challenge, check that TXT records are created:
+
+```bash
+# Check for _acme-challenge TXT records
+dig +short _acme-challenge.yourdomain.com TXT
+
+# Or using Azure CLI
+az network dns record-set txt list --resource-group YOUR_RG --zone-name yourdomain.com
+```
+
+## Troubleshooting
+
+### Issue: "Failed to get Azure access token"
+
+**Solution:** Verify your Azure credentials are correct and the Service Principal has the necessary permissions.
+
+### Issue: "Failed to create TXT record"
+
+**Solution:** 
+- Verify the Service Principal has DNS Zone Contributor role
+- Check that the resource group and zone name are correct
+- Ensure the Azure API version is supported
+
+### Issue: DNS propagation timeout
+
+**Solution:** 
+- Wait longer for DNS propagation (may take more than 60 seconds in some cases)
+- Verify your DNS zone is properly configured
+- Check that nameservers are responding
+
+## Dry Run Testing
+
+To test without actually requesting certificates from Let's Encrypt:
+
+1. Use the Let's Encrypt staging environment in your dehydrated config:
+   ```bash
+   CA="https://acme-staging-v02.api.letsencrypt.org/directory"
+   ```
+
+2. Test with a test domain or subdomain that won't impact production
+
+## Success Criteria
+
+A successful test should:
+
+1. ✓ Accept the challenge type parameter correctly
+2. ✓ Select the appropriate hook script
+3. ✓ Create TXT records in Azure DNS (for dns-01)
+4. ✓ Successfully validate the domain
+5. ✓ Clean up TXT records after validation
+6. ✓ Generate valid SSL certificates
+7. ✓ Post results to Slack/Teams (if configured)
+
+## Syntax Validation
+
+Before running tests, validate script syntax:
+
+```bash
+# Check main script
+bash -n letsencrypt_drupal.sh
+
+# Check Azure DNS hook
+bash -n hooks/azure_dns_hook.sh
+
+# Run shellcheck for code quality
+shellcheck -x letsencrypt_drupal.sh
+shellcheck -x hooks/azure_dns_hook.sh
+```
+
+## Additional Notes
+
+- The default behavior (http-01) should work exactly as before
+- The dns-01 challenge requires no Drupal module but needs Azure DNS access
+- Both challenge types can coexist - select the appropriate one per execution
+- All existing http-01 functionality is preserved

example_project_config/letsencrypt_drupal/config_undp_azure.env.sh

@@ -0,0 +1,38 @@
+#!/usr/bin/env bash
+
+# Example Azure DNS configuration for dns-01 challenge
+# Copy this file and adjust values for your Azure environment
+
+# Slack/Teams endpoint and target channel (optional).
+# Get it here: https://my.slack.com/services/new/incoming-webhook/
+SLACK_WEBHOOK_URL='https://hooks.slack.com/services/XXXXXXXXX/XXXXXXXXX/XXXXXXXXXXXXXXXXXXXXXXXX'
+SLACK_CHANNEL='CHANNEL-NAME'
+
+TEAMS_WEBHOOK_URL=''
+
+# UUID of target environment for cert deploy.
+# Easiest to get from URL in Acquia Cloud UI. See https://cloudapi-docs.acquia.com/#/Environments/getEnvironment
+# (Second uuid in URL when looking at specific environment.)
+CERT_DEPLOY_ENVIRONMENT_UUID="XXXXXX-XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX"
+
+# Azure DNS Configuration for dns-01 challenge
+# These should be stored securely and sourced from a secrets file
+# For Acquia Cloud, store in /mnt/files/PROJECT.ENV/secrets.settings.php or similar
+
+# Azure Subscription ID
+export AZURE_SUBSCRIPTION_ID="XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX"
+
+# Azure Active Directory Tenant ID
+export AZURE_TENANT_ID="XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX"
+
+# Service Principal Client ID (Application ID)
+export AZURE_CLIENT_ID="XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX"
+
+# Service Principal Client Secret
+export AZURE_CLIENT_SECRET="YOUR-CLIENT-SECRET-HERE"
+
+# Resource Group containing the DNS Zone
+export AZURE_RESOURCE_GROUP="your-resource-group-name"
+
+# DNS Zone name (e.g., example.com)
+export AZURE_DNS_ZONE="example.com"