MongoDB 3.6 Change Streams - AWS Lambda demo

This repository demonstrates how to use MongoDB Change Streams as a trigger to an AWS Lambda function.

AWS Lambda function

This is the Lambda function used to send an email from a MongoDB Change Stream notification

Configuration

1. In AWS, create an S3 bucket, create a Templates folder and upload the ChangeStreamInfo.html file into it.
2. In AWS, configure SES with (at least) 2 verified email addresses (one will be used to send the email, the other to receive the email)
3. In sam.yaml, customize
   • the S3_BUCKET environment variable with the value of your S3 bucket
   • the FROM_ADDRESS environment variable with the value of the first SES-verified email address
4. In config.js set the toEmail property to your other SES-verified email address

Test

1. Install SAM Local on your machine
2. In the email-lambda folder, run npm install to install the required Node dependencies
3. Run sh invoke.sh from Terminal to test your Lambda function local

Deployment

1. Run sh package.sh from Terminal to package your SAM package
2. Run sh deploy.sh from Terminal to deploy your SAM package to AWS
3. Once the script has completed (without errors), sign in into the AWS Console
4. Navigate to the IAM service, edit the ChangeStream-SendEmail-SendEmailRole-[random] role
5. Add 2 inline policies
   • For Amazon S3, add the GetObject policy on the * ARN
   • For Amazon SES, add the SendEmail policy
6. Test that the SendEmail lambda function works correctly (you can use the event.json file as a test event).
7. Navigate to the API Gateway console and select API Keys. Create an API Key and copy the value to the awsApiGWKey property of the trigger/config.js file.
8. Select the ChangeStream-SendEmail API and in the Actions menu, select Deploy API for the Stage deployment stage.
9. In the ChangeStream-SendEmail/Stages menu, select the Stage stage and copy the Invoke URL value. Copy it to the awsApiGWUrl property of the trigger/config.js file.

MongoDB 3.6 cluster configuration

You must configure a replica set for change streams to work. Follow the instructions below:

1. Make sure the latest version of mtools is installed on your machine
2. In the trigger folder, edit setup.sh and update the MONGO_LOCATION variable to point to your local MongoDB 3.6 installation root
3. Run sh setup.sh to set up your MongoDB 3.6 replica set using mlaunch
4. Run sh run.sh to start your MongoDB 3.6 replica

AWS Lambda trigger configuration (with Change Streams)

1. In the trigger folder, run npm install to install the required Node dependencies
2. Run node trigger.js to start listening to change streams coming from the devices collection of the demo database.
3. In a separate Terminal window, run node addDoc.js - this will add a document to the devices collection of the demo database.
4. Look at the window running trigger.js and if everything was properly configured you should get a Change Stream output log message, followed by a message confirming that the email was successfully sent.
5. Check your email inbox and you should find an email titled MongoDB 3.6 Change Stream notification for ecobee_1234.