Challenge Overview
Topcoder technology heatmap is a single page app that displays an overview of various technologies and their "attractiveness" on Topcoder. The current app (see here) uses a static google sheet to get the data for the heatmap. In one of the challenges at TCO 2016 the members were tasked with creating a backend that will generate this data dynamically from two data sources Topcoder Looker and Stackoverlow API. In this challenge we will use the winning submission (attached in the forums) and modify it so it can be used by the heatmap application. We will use AWS Lambda functions to create a backend with serverless framework
The first requirement is to create a daily job that will generate data for all technologies and push it to Elasticsearch (so we'll have historical data in ES). The job should be implemented as an AWS Lambda function and it should be scheduled to run once every day. If the job fails due to IO error, it should be retried up to three times with a backoff timer. You can create an elasticsearch instance in the free AWS tier for testing. There is a bug in the challenge submission for entries in the looker data when technology_name is null and fixing that bus is in the scope (just filter out those records)
Final requirement is to add a lambda function that will return heatmap data in this format: {date, [{name, communityReady, widelyUsed, crowdReady, shouldSell, type}] }. We don't yet have data for crowdReady, shouldSell and type fields, so return "N/A" by default. Data should be fetched from elasticsearch. Lambda should have a query parameter 'date' that will be used to select proper data from elasticsearch. In case date isn't provided, return the latest data from ES. If ES does not have data for that specific date, return the data for the closest date.
You can use serverless-offline for testing the lambdas locally, but you should also deploy them to AWS and provide Postman collection for testing.
Heatmap application source code is available here (just for reference, you don't need it in this challenge) > https://github.com/cwdcwd/operation-predator
Submit deployment/verification guide
Submit Postman collection.
The first requirement is to create a daily job that will generate data for all technologies and push it to Elasticsearch (so we'll have historical data in ES). The job should be implemented as an AWS Lambda function and it should be scheduled to run once every day. If the job fails due to IO error, it should be retried up to three times with a backoff timer. You can create an elasticsearch instance in the free AWS tier for testing. There is a bug in the challenge submission for entries in the looker data when technology_name is null and fixing that bus is in the scope (just filter out those records)
Final requirement is to add a lambda function that will return heatmap data in this format: {date, [{name, communityReady, widelyUsed, crowdReady, shouldSell, type}] }. We don't yet have data for crowdReady, shouldSell and type fields, so return "N/A" by default. Data should be fetched from elasticsearch. Lambda should have a query parameter 'date' that will be used to select proper data from elasticsearch. In case date isn't provided, return the latest data from ES. If ES does not have data for that specific date, return the data for the closest date.
You can use serverless-offline for testing the lambdas locally, but you should also deploy them to AWS and provide Postman collection for testing.
Heatmap application source code is available here (just for reference, you don't need it in this challenge) > https://github.com/cwdcwd/operation-predator
Final Submission Guidelines
Submit source code for lambda functionsSubmit deployment/verification guide
Submit Postman collection.