Challenge Overview
Challenge Objectives
-
Create a script to migrate challenge data from our legacy database into our new V5 API.
Tech Stack
-
Node.js
-
Swagger
-
DynamoDB
-
ElasticSearch
Project Background
In this series of challenges, we will build the version 5 (V5) of the challenge API.
Code Access
Challenge API
Repo: https://github.com/topcoder-platform/challenge-api
Branch: develop
Resources API
Repo: https://github.com/topcoder-platform/resources-api
Branch: develop
Legacy API
Repo: https://github.com/appirio-tech/ap-challenge-microservice
Branch: develop
Detailed requirements
As part of this challenge, we need your help to create a migration script using Node.js that will be used to migrate existing challenge data from our legacy system into the new Challenge API V5.
Challenge API changes
In order for the migration script to work, you will have to make the following modifications to the challenge API:
-
Add a winners array to the challenge model that is going to be used for storing the challenge winners in the following format: [{ userId: 12345678, handle: 'thomaskranitsas', placement: 1 }]
Migration script
This is a new CLI that will be used to read data directly from our legacy Informix DB and it will insert that data into our new DynamoDB as well as our ElasticSearch.
The migration script should:
-
Read directly from the InformixDB, format the challenge data into the proper form to match with the model of the V5 Challenge API and insert the data directly into our DynamoDB and ElasticSearch. You may copy the logic for that from the challenge API repo.
-
The new data depends on entities (eg challenge type) with UUIDs while the legacy data may have plain string values. For such properties, you need to create a configurable mapping to map the old with the new values.
-
Verify that a challenge with the same legacyId does not exist before inserting the new challenge record. This will ensure we won’t have duplicate entries in case the script fails and we need to run it again.
-
Show some progress (eg the number of migrated challenges and/or the IDs of the successfully migrated challenges)
-
If, for any reason, the script fails to migrate a challenge, it should log the challenge ID and the error details into a JSON file. It can be a single JSON file with an array of failed attempts or multiple JSON files, one per failed attempt. Up to you.
-
Have a way to retry migrating just the failed challenges. For example, an npm run retry script that will read the JSON file(s) from the requirement above and will retry migrating the challenges with those IDs. -
Overwrite the created/updated fields to match the ones from the legacy challenge data.
Should you have any questions, feel free to ask on the challenge forum!
What to Submit
Submit a zip file with a git patch file for the latest commit in the develop branch of the challenge API and the whole project folder for the migration script.
Make sure to mention the exact commit you used so we can apply your patch without issues and a verification document with details on how to verify your submission.