AWS DynamoDB Streams | Lambda | Real time data processing using DynamodDB Streams & Lambda | Demo

Real time data processing using AWS DynamodDB Streams and Lambda

STEP 1: create index.js file with following content:

console.log('Loading function');

exports.handler = function(event, context, callback) {
console.log(JSON.stringify(event, null, 2));
event.Records.forEach(function(record) {
console.log(record.eventID);
console.log(record.eventName);
console.log('DynamoDB Record: %j', record.dynamodb);
});
callback(null, "message");
};

STEP 2: compress the index.js using any compress utility:
jar -cfM function.zip index.js

STEP 3: IAM role for entity AWS Lambda, role name: lambda-dynamodb-role and policy with permission: AWSLambdaDynamoDBExecutionRole

STEP 4: run Lambda create-function CLI command with IAM role ARN:
aws lambda create-function --function-name ProcessDynamoDBRecords --zip-file fileb://function.zip --handler index.handler --runtime nodejs8.10 --role arn:aws:iam::392821317968:role/lambda-dynamodb-role

STEP 5: invoke lambda function to test
aws lambda invoke --function-name ProcessDynamoDBRecords --payload file://input.txt outputfile.txt

STEP 6: create DynamoDB table with following details:
Table name – lambda-dynamodb-stream
Primary key – id (string)

STEP 7: enable DynamoDB table stream

STEP 8: create event mapping of Lambda function with stream:
aws lambda create-event-source-mapping --function-name ProcessDynamoDBRecords --batch-size 100 --starting-position LATEST --event-source arn:aws:dynamodb:ap-south-1:392821317968:table/lambda-dynamodb-stream/stream/2019-06-29T12:39:47.680

STEP 9: list event mapping to verify:
aws lambda list-event-source-mappings

or list for specific function:
aws lambda list-event-source-mappings --function-name ProcessDynamoDBRecords