Feature: SNS Topic Fan-Out L3 Construct with Typed Publishers and Subscriptions

[hoegertn]

Summary

Add an opinionated L3 construct for Amazon SNS topics that follows the cdk-serverless code generation approach. Users define topic message schemas and subscription configurations in a definition file and get typed publishers, managed subscriptions with filter policies, DLQ on delivery failures, and Lambda subscriber handlers — all generated automatically.

Problem

SNS is the go-to service for fan-out (one event → multiple consumers) in serverless architectures, but the setup involves:

• Topic creation with encryption and access policies
• Subscription management across different protocols (SQS, Lambda, HTTP, Email)
• Filter policy configuration to route message subsets to specific subscribers
• DLQ configuration on subscriptions for delivery failure handling
• IAM permissions for publishing and subscribing
• No typed message payloads — publishers construct raw PublishCommand calls manually
• No type-safe filter policies — filter expressions are hand-written JSON

For the common case of "publish a typed message, fan out to N subscribers with filtering," this is too much ceremony.

Proposed Solution

Definition File

# topics/notifications.yaml
publisherName: Notifications
topics:
  OrderNotifications:
    schema:
      type: object
      properties:
        eventType:
          type: string
          enum: [order_placed, order_shipped, order_delivered, order_cancelled]
        orderId:
          type: string
        customerId:
          type: string
        amount:
          type: number
        carrier:
          type: string
      required: [eventType, orderId, customerId]
    messageAttributes:
      - eventType    # available for filter policies
      - customerId
    subscriptions:
      ShippingProcessor:
        protocol: lambda
        filter:
          eventType: [order_placed]
      DeliveryTracker:
        protocol: lambda
        filter:
          eventType: [order_shipped, order_delivered]
      CustomerEmailer:
        protocol: lambda
        filter:
          eventType: [order_placed, order_shipped, order_delivered, order_cancelled]
      AnalyticsQueue:
        protocol: sqs
        # no filter — receives everything

  SystemAlerts:
    schema:
      type: object
      properties:
        severity:
          type: string
          enum: [info, warning, critical]
        service:
          type: string
        message:
          type: string
      required: [severity, service, message]
    messageAttributes:
      - severity
    subscriptions:
      OpsChannel:
        protocol: lambda
        filter:
          severity: [warning, critical]
      AllAlertsArchive:
        protocol: sqs

Projen Integration

import { TopicPublisher } from 'cdk-serverless/projen';

new TopicPublisher(project, {
  publisherName: 'Notifications',
  definitionFile: 'topics/notifications.yaml',
});

Running projen generates:

• Typed message interfaces per topic (e.g. OrderNotificationMessage, SystemAlertMessage)
• Typed handler signatures for each Lambda subscription (e.g. ShippingProcessorHandler)
• Typed message attribute interfaces for filter policies
• A typed TopicPublisher class with per-topic publish methods
• The L3 CDK construct

CDK Construct Usage

import { NotificationsTopicPublisher } from './generated/topic.notifications.generated';

const topics = new NotificationsTopicPublisher(this, 'Topics', {
  singleTableDatastore, // optional
  additionalEnv: {
    EMAIL_SERVICE_URL: props.emailServiceUrl,
  },
});

// Expose the SQS queue references for further wiring
const analyticsQueue = topics.orderNotifications.subscriptions.analyticsQueue.queue;

The construct automatically:

• Creates SNS topics with SSE encryption (KMS or SNS-managed)
• Creates Lambda functions for each Lambda subscription
• Creates SQS queues for each SQS subscription (with DLQ)
• Configures filter policies derived from the definition file
• Sets up DLQ on each subscription for delivery failures
• Sets up raw message delivery for SQS subscriptions
• Grants sns:Publish to Lambdas using the generated publisher
• Integrates with the existing monitoring infrastructure (delivery failures, DLQ depth)

Handler DX

// Lambda subscription handler — typed message, typed attributes
export const handler: ShippingProcessorHandler = async (message, attributes) => {
  // message is typed as OrderNotificationMessage
  const { orderId, customerId, amount } = message;
  // attributes.eventType is guaranteed to be 'order_placed' due to filter
  
  await initiateShipping(orderId, customerId);
};

Publishing DX

import { NotificationsPublisher } from './generated/topic.notifications-publisher.generated';

const publisher = new NotificationsPublisher();

// Type-safe — schema enforces payload and required message attributes
await publisher.publish('OrderNotifications', {
  message: {
    eventType: 'order_shipped',
    orderId: '123',
    customerId: 'cust-456',
    amount: 99.99,
    carrier: 'DHL',
  },
  // messageAttributes auto-extracted from the message based on definition
});

await publisher.publish('SystemAlerts', {
  message: {
    severity: 'critical',
    service: 'payment-gateway',
    message: 'Payment provider timeout rate above 5%',
  },
});

Integration Points

• EventBus construct: EventBridge rule → SNS for fan-out (EventBridge for routing, SNS for delivery)
• QueueProcessor construct: SNS → SQS subscription feeds into a QueueProcessor for buffered processing
• SingleTableDatastore: Lambda subscription handlers get a pre-configured datastore client
• RestApi / GraphQlApi: API handlers use the generated publisher to broadcast notifications
• RealtimeApi (if implemented): Lambda subscriber publishes to AppSync Events for client push

Differences from EventBus Construct

Concern	EventBus	TopicPublisher
Primary pattern	Event routing (1 event → 1 matched handler)	Fan-out (1 message → N subscribers)
Filtering	EventBridge rules (pattern matching on full event)	SNS filter policies (message attributes)
Subscriber types	Lambda (via rule target)	Lambda, SQS, HTTP, Email, SMS
Use when	Routing events to specific services	Broadcasting to multiple consumers
Ordering	No ordering guarantees	FIFO topics available

Both can coexist — a common pattern is EventBridge for inter-service routing and SNS for intra-service fan-out.

Out of Scope

• SNS → HTTP/HTTPS endpoint subscriptions with confirmation handling → complex, different use case
• SNS → Email/SMS subscriptions → useful but no code generation benefit
• SNS mobile push (Platform Applications) → different domain
• Cross-account topic subscriptions → future enhancement
• FIFO topics → future enhancement (add when demand exists)