Challenge Overview
Challenge Objectives
-
Synchronize DynamoDB data to Elasticsearch - Lambda function
Project Background
-
Topcoder Member service is a service that manages a lot of member related data - user profile, traits, stats, search etc
-
In this challenge we want to create a lambda function that synchronizes data in dynamoDb and Elasticsearch
-
Future challenges will implement data loading and frontend changes in our community app
Technology Stack
-
NodeJS
-
DynamoDB
-
Elasticsearch
Code access
Create a new project structure for the lambda function. Sample database structure for the new DynamoDB table is posted in the forums. Project repository is at https://github.com/topcoder-platform/member-stats-sync-lambda
Individual requirements
At the moment, member statistics are calculated for each user and persisted into a DynamoDB table. However, members can be in multiple “groups” and we want to store member statistics for each group separately.
On a data storage level, the change will be simple - creating a table in DynamoDB with a composite primary key - (userId, groupId).
Our task in this challenge is updating Elasticsearch index whenever a record in DynamoDB is updated/created. DynamoDB Streams should be used to trigger a Lambda function that will just update the record in ES (same record structure as DynamoDB record).
Update Readme.md with details on how to deploy the lambda function (first time deploy and updating on code change) and how to configure the DynamoDB trigger (manual configuration). Also, create instructions for deploying and configuring the lambda using serverless framework
General requirements
-
Unit tests are in scope.
-
Sensitive configuration parameters should be set from environment variables (DB URL, AWS credentials, etc).
What To Submit
-
All source code
-
Verification and configuration guide