Topcoder v5 Lookup API | A tale of two apps

Challenge Objectives

• Extract Elasticsearch related code from our existing app and create a new app

 

Project Background

The lookup api aims to provide an api to be used across Topcoder’s front end apps that provide a consistent set of results for actions such as looking up countries, educational institutions and others.

 

Technology Stack

• Nodejs version 10

• Elasticsearch version 7

 

Code access

You can find the existing code here. Use the develop branch as your base branch

 

Individual requirements

Extract Elasticsearch related code from our existing app and create a new app (Major Requirement)

• The lookup api at the moment is a single code base, where we integrate with both Elasticsearch and Dynamodb.

• We would like to split this into two apps. The current app will integrate with Dynamodb, but its integration with Elasticsearch will be limited to GET requests only. That is, only for the GET request, we will be interfacing with Elasticsearch, as we do currently. (Note that currently, the app will first get the data from Elasticsearch, and then fall back to Dynamodb. This behaviour will NOT change in this contest)

• For all other requests, the current code will not interact with Elasticsearch. It will only interact with Dynamodb, wherever applicable. If there is a create, update or delete operation, after interfacing with dynamodb, you will make use of the Topcoder Bus API Wrapper module to post an event to the bus api (kafka). Each event will have a different topic, details of which will be shared in the contest forum

• Next, you will create a separate app that will listen for events on kafka. These are the same event topics to which the previous app will post to.

• Depending on the event, the new app will carry out the necessary actions in Elasticsearch, namely create, update and delete.

• So, the new workflow will be as follows:

  • The API receives a create / update / delete action

  • The API carries out the action in Dynamodb

  • The API will then post an event to the bus api with the necessary details

  • The new app will be listening to the same event topics, as used by the current API in kafka

  • Based on the event, it will create / update / delete data in Elasticsearch

• The API needs to post all relevant details necessary for the new app to carry out the relevant action in Elasticsearch. For reference, you can use the submission api (that integrates with dynamodb, and for GET requests only, it integrates with Elasticsearch) and the submission elasticsearch processor (that only integrates with Elasticsearch).

 

Implement a health check endpoint (Minor Requirement)

• The current code base has a health endpoint. The new app will also have a health endpoint. You can make use of the topcoder health check dropin module for the new app.

• Check out the one present on submission elasticsearch processor that you can use as reference

 

Deployment guide and validation document

Update the current api's deployment guide as needed for this contest. No verification document necessary.

For the new app, provide a new deployment guide. No verification document necessary.

 

Important Notes

• Follow existing conventions used in the app

• Please don’t provide any local instance of Elasticsearch through docker.

• Please don’t provide any scripts to populate the data or any helper scripts. Please focus on the requirements mentioned in this specification.

• You need to remove and move the relevant tests to the new app too.Reviewers will use the dedicated test section in the scorecard when considering issues related to tests

Final Submission Guidelines

For changes to the api / current code base, you can submit either the git patch or the full code base. For the new app, zip the source code. Zip both the code base and upload to Topcoder as a single zip file.