Running a flowcraft sqs adapter worker on a lambda #6
-
|
I'm working on setting up a distributed adapter using SQS, and I see in the docs that the SQS adapter can be used in workers on EC2, fargate, and lambda, so I was wondering what a lambda setup would look like. The SQS adapter looks like it's meant to be used in a long running worker service that polls the SQS queue, but ideally I'd want the lambda to handle the SQS polling through a trigger, and just use the adapter for running the node and context management. If this is possible, please let me know, thanks |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment
-
|
Yes, this is now possible. You can use the SQS adapter's Here's what it looks like: // lambda/workflow-worker.ts
import type { SQSEvent, SQSHandler } from 'aws-lambda'
import { DynamoDBClient } from '@aws-sdk/client-dynamodb'
import { SQSClient } from '@aws-sdk/client-sqs'
import { DynamoDbCoordinationStore, SqsAdapter } from '@flowcraft/sqs-adapter'
const sqsClient = new SQSClient({ region: process.env.AWS_REGION })
const dynamoDbClient = new DynamoDBClient({ region: process.env.AWS_REGION })
const coordinationStore = new DynamoDbCoordinationStore({
client: dynamoDbClient,
tableName: process.env.COORDINATION_TABLE!,
})
const adapter = new SqsAdapter({
runtimeOptions: {
blueprints: { /* your blueprints */ },
registry: { /* your node implementations */ },
},
coordinationStore,
sqsClient,
dynamoDbClient,
queueUrl: process.env.SQS_QUEUE_URL!,
contextTableName: process.env.CONTEXT_TABLE!,
statusTableName: process.env.STATUS_TABLE!,
})
export const handler: SQSHandler = async (event) => {
for (const record of event.Records) {
const job = JSON.parse(record.body || '{}')
await adapter.handleJob(job)
}
}Set your SQS event source mapping to How it works:
No polling, no persistent worker, just bring the orchestration logic. I added this same serverless pattern to the GCP (Cloud Functions + Pub/Sub), Azure (Functions + Queue Storage), and the new Vercel and Cloudflare adapters. Docs have been updated across the board. |
Beta Was this translation helpful? Give feedback.
Yes, this is now possible. You can use the SQS adapter's
handleJob()method directly in a Lambda handler without the polling loop. The Lambda + SQS event source mapping handles the polling for you, and the framework handles node execution, context management, fan-in coordination, and successor job enqueueing.Here's what it looks like: