[Repost] Topcoder MM Processor | Automate deployments to EC2 instance

Challenge Objectives

• Use AWS services to deploy source code located in Amazon S3 to an EC2 instance

 

Project Background

• When Topcoder runs contests in the Marathon Match track, they involve a “processor” that takes care of executing the submissions uploaded by participants and creates a review for them based on the score received.

• This process is currently carried out manually. The Copilot or PM for that contest set up the processor (fine tune the configurations specific for that contest), launch the processor, monitor the deployment and execution of the submissions etc.

• This contest aims to automate this process

 

Technology Stack

• Nodejs version 10

• Amazon Web Services

• Gitlab

 

Code access

We will share the current source code with your in the contest forum. This source code has two parts - one is the main source code, that is the Marathon Match processor. The other is the files related to auto deployment of the processor code from Gitlab to Amazon S3.

 

Individual requirements

Use AWS services to deploy source code located in Amazon S3 to an EC2 instance (Major Requirement)

• Our current pipeline / workflow is like this:

  • CoPilot will update a Dockerfile in the source code, commit their changes and push to Gitlab

  • Gitlab will trigger the pipeline that will then build the Dockerfile, push the image to Dockerhub and update a configuration in the same code.

  • It will then zip the code and copy it over to Amazon S3.

• This is where we will be starting from for this contest. Once the code is in S3, we would like you to carry out steps that will deploy the code to an EC2 instance.

• We believe that you will need to make use of AWS CodePipeline, along with CodeBuild and CodeDeploy, but we are open to suggestions here.

• Here are the steps that we are currently carrying out, once we have our EC2 instance ready, and the code cloned over:

  • We install pm2, the nodejs process manager.

  • We install the dependencies by running npm install

  • We then start the app using the pm2 start command.

  • Note that all the necessary configuration are already set up (in the ecosystem.config.js file), which is copied over from Gitlab.

• You will be starting off with the source code in Amazon S3. Note that this is in zip format. From there, you need to end up with an EC2 instance that has copied the code over and the app deployed.

• Note that the source code in Amazon S3 can get updated, in which case your workflow must trigger again. The running instance of the app in EC2 must gracefully exit and be replaced with the new instance.

 

Deployment guide and validation document

A well written deployment guide is essential for the success of your solution. Reviewers should be able to deploy your solution by following the deployment guide alone. Assume the end user is familiar with command line interface and can navigate through the AWS interface - your deployment steps need to be detailed.

 

The provided source code contains a README.md file. In this we have a dedicated section for Gitlab CI / CD. You need to update this section with the steps relevant for this contest.

 

Important Notes

• In the provided source code, the folder aws contains all the aws deployment related files. You can create sub folders in this folder and organize the code properly. Keep all your deployment related files in the aws folder only.

• The AWS EC2 instance will be accessible only through VPN. You cannot directly ssh into the EC2 server

• Please provide the license details for any templates that you are using. Kindly ensure that the licenses are in the approved list of licenses for use in Topcoder Contests. This is a minor requirement.

 

A Note on the Review

• We will be using the Subjective scorecard for this contest

• Your score will be based on:

  • How easy it is to follow your deployment guide

  • How easy it is to deploy your solution

  • How much automated your solution is. We are looking for a solution with the least effort involved at our end to set it up.

• You will get a passing score only if your solution can be used immediately (and meets the requirements) without requiring any major changes. Reviewers, note that getting a score more than or equal to 50 is a passing score. Your solution should be worth the prize money.

Final Submission Guidelines

Since you are not making any changes to the source code, you do not have to upload it. Instead, zip only those files needed to make this work along with your deployment guide and describe where to place these files in the source code (at the ROOT folder, under a specific folder etc).Upload your zip file to Topcoder.