Legacy Submission Processor | Initial Setup

Challenge Objectives

• Setup Informix database

• Create a processor that listens for a specific event on kafka

• Based on the message received, the processor will carry out specific actions

 

Project Background

• Legacy submission processor is an existing app that we are looking at rewriting

• It consists of three modules / repositories:

  • Legacy Submission Processor

  • Legacy Marathon Match Processor

  • Legacy Processor Module

• The processor would essentially update the details of a submission in an Informix database, which is used by other processors elsewhere

• After the update, it would then update the submission details in our submission api

• This project aims to bring together all three repositories into a single repository with changes made in the workflow

 

Technology Stack

• Nodejs version 10

• Kafka version 2

 

Code access

This is the first contest in the series and thus there is no code to share. You will be writing the code from scratch. The 3 repositories shared above are only meant for reference

 

Individual requirements

Setup Informix database (Major Requirement)

• The Legacy Marathon Match Processor module has a Validation.md file that describes how you can deploy the entire app using docker. This deployment also deploys the dependencies of the app such as Kafka and Informix database.

• Extract the Informix database deployment from these steps / docker related files and create a new Informix only deployment using docker

• You will use this when working with the app

• Note that you also need to provide scripts to insert test data into the database, data which will be used by the processor as explained in the next requirement. This step has also been defined in the Validation.md file, which you can reuse.

 

Create a processor that listens for specific events on kafka (Major Requirements)

• Create a nodejs app that listens to the “submission.notification.aggregate” topic in kafka

• The full structure of the message expected can be found here.

• Only process messages where the “originator” property of the message is “submission-api”

• Even in this case, process only those messages where the resource is “submission” (message.payload.resource === “submission”) and the original topic of the message is “submission.notification.create (message.payload.originalTopic === “submission.notification.create”)

• Thus, you are only going to process the submission creation event.

 

Based on the message received, the processor will carry out specific actions (Major Requirement)

• Once the message is ready for processing, the app needs to create the submission in the Informix database using the details present in the message.

• You will have to create a service within the app to interface with the Informix database.

• This service, in the process of creating the submission in the Informix database will carry out the following tasks:

  • First get details of the challenge associated with the submission. These details are fetched from the Informix database itself, which is why it is essential that during your development, you provide scripts to populate the database with data, which the reviewers can make use of (and you can make use of too during your development)

  • You then need to generate a new ID for that submission (called as legacy upload id)

  • When fetching the details of the challenge earlier, you would get info on the current phase of the challenge. Based on the current phase the challenge is in, we determine if the submission is a Final Fix submission or not. If not, then we generate another ID called legacy submission id for it. We also make a note of the upload type in either case.

  • You then convert the audit related timestamps of the submission from UTC (which is the format the dates are in, as received from Kafka) to ET (the timezone of Topcoder)

  • Then insert the submission in Informix

  • Next, if the submission is a Final Fix submission, we proceed to prepare the PATCH object for use with the Submission API. Else, we have two more insertions into the Informix database(Insertion 1 and Insertion 2). And then finally we prepare the PATCH object.

  • Once the PATCH object is ready, we proceed to PATCH the submission using the Submission API. Here, you will NOT use the submission api directly. Instead, you are expected to use the submission api wrapper.

  • What we have described above is basically the workflow of the AddSubmission function, which you will rewrite for this contest.

• Note. The instructions above are what your solution needs to do. We have provided references to the current code as well, where applicable. However, you will NOT follow every step of the current code (because we are skipping / changing the workflow slightly). Thus, it is important that you follow only those instructions mentioned above and NOT carry out everything from the existing code base. (Example - current code determines the contest type, it also determines if multiple submissions for the same contest are allowed or not and other tasks - which are NOT part of the tasks for this contest). Please only provide the tasks as described in the above steps.

 

Deployment guide and validation document

Provide a deployment guide in markdown format with instructions on how to deploy your app. In the deployment guide, kindly provide a section named Verification with steps on how to verify that your solution meets the requirements. 

 

Important Notes

• Use async await pattern

• Use standard library as your linter

• Tests are in scope. Use mocha and chai for your tests. Provide both unit and e2e tests with a code coverage more than 80%

• For your submission, we expect a single app and not multiple apps. The interaction with Informix database as well as with kafka - both will be part of the same codebase.

• Again - you will NOT provide the same workflow as present in the current codebase(s). You are writing a new codebase from scratch, taking bits and pieces from the old app. Stick to the tasks as defined in the specification

Final Submission Guidelines

Zip your solution and upload it to Topcoder