[72h] Topcoder Plataform - Final Fixes to Legacy Migration Script

Challenge Objective

• Add date parameter to the script and save last run date
• ���Add challenge type table

Background

• Topcoder has developed a series of new APIs to better support all of their web sites and systems
• These new APIs make up the version 5 API
• Most of these APIs make use of Elastic Search for their GET calls to speed up searching the data

Technology Framework

• Node.js
• Elastic Search
• DynamoDB

Requirements

• Update the legacy migration script code to add a date parameter to the script. The time the script was executed should be stored in DynamoDB (ChallengeHistory table). The idea is avoid pulling the entire challenges everytime we run the script.
• This date parameter should be used to fetch challenges and resources where the create date is greater than this date.
• If no date is passed the script must:
  • Check the DymanoDB for last run date and use that date
  • If no date exists in DymanoDB throw error and ask for date parameter
  • After running the script the current date should be stored in DynamoDB as the last time the script was executed
• If date is passed the script must:
  • Ignore DymanoDB last run date and use the date parameter
  • After running the script the current date should be stored in DynamoDB as the last time the script was executed
• The DynamoDB should hold the run date and the statistics: how many challenges and resources where added. You should always insert to this DynamoDB table so we have the statistics for each execution. The last run date is the most recent record.
• Retry action is not changed
• Create ChallengeType table. The idea is to load the challenge types only once (see ResourceRole for example). Should create index in ES and search using ES the same way as ResourceRole. 
• ChallengeType records will come from https://api.topcoder.com/v4/challenge-types:
  • The ID in the response should be converted to legacyId property and ID in record stored in DynamoDB should be auto-generated
  • Update the challenge migration script to search for the challenge type using legacyId in DynamoDB and store in Challenge table the id generated above (see ResourceRole for example)
• Challenge type mapping in configuration file should not be used anymore
• Update the create and drop scripts to create/drop ChallengeHistory and ChallengeType tables
• Update the view data to view ChallengeHistory and ChallengeType tables
• Update the view es data to view ChallengeType data in ES
• Legacy migration script code:
  • https://github.com/topcoder-platform/legacy-challenge-migration-script/tree/develop

Final Submission Guidelines

• Patch to repository (develop branch)