Topcoder Taas API - Elastic search Processor

Challenge Objectives

Welcome, in this challenge we will update TaaS API that we have developed for the new app that will be used to manage topcoder TaaS(Gig Work)

Project Background

We’re building a new APP to manage Topcoder Talent as a Service offering. That includes creating new job postings, members applying to jobs, customers reviewing job applications and finally assigning members to jobs and managing feedback from both clients and members. This application will have microservice as the backend API and frontend developed with React JS.

 

In the previous challenge, we added ElasticSearch(ES) for better performance in the codebase. Now, in this challenge, we need to move that elastic search to processor-based similar to u-Bahn ES processor https://github.com/topcoder-platform/u-bahn-es-processor 

 

Technology Stack

NodeJS, PostgreSQL, Sequelize, Express, Elastic Search

Code access

The codebase is shared on the forum.

Individual requirements

We have two major requirements for this challenge:

1. Create a separate processor to handle to store the data on ElasticSearch(ES) similar to the u-Bahn es processor.
2. Update the bookings service APIs to post the data to the processor

So after these changes, the new flow will be as:

1. The user calls the Create/update/delete API then 
   1. bookings service will create in the database as of now then instead of saving to ES directly it will post the message to Kafka using Topcoder bus API
   2. The processor will update the record on the ES
2. For GET /:id APIs, please add a flag `fromDb` which if true returns from the database else will get the data from ES. see here
3. For search methods, it should try to fetch the records from ES, if failed then return from the database with flag `fromDb` as true. See here

Health Check

Please implement a health check on both taas API codebase and new taas-es-processor. For this please use library topcoder-healthcheck-dropin 

1. For Processor please see the sample here https://github.com/topcoder-platform/challenge-processor/blob/develop/src/app.js#L66 and https://github.com/topcoder-platform/u-bahn-es-processor/blob/develop/src/app.js#L113 
2. For API, configure the topcoder-healthcheck-dropin to use as middleware with the below check. 
   1. It should check if the database is functioning properly or not by pooling DB with some a simple query such as `SELECT NOW() AS "the_time"`. Make sure the connection is closed after the query. Sequelize doesn’t close the connection automatically. For this case(health check), please close the connection after checking(both error/success case) so that performance won’t be hit.

General for all requirements

• Use the controllers/services model for the new API. 
• Support both local and cloud ES, similar to here 
• Use Topcoder BUS API wrapper to post the message to the Kafka
• Kafka topics should be configurable 
• Unit tests with a minimum of 90% coverage on both source code are required
• Logging is required for all services. 
• Unit tests are required. Test Coverage shouldn’t be degrade
• Update swagger if you’re making any changes to API definition in this challenge, and create/update Postman scripts for validation if required. 
• NOTE: Logging standards document is posted in the forums - update all the existing endpoints to match the logging standard.
• Please follow the https://github.com/topcoder-platform/u-bahn-es-processor very closely while creating the processor. In place of u-Bahn, we will use `taas`
• Make sure no functionality is broken on altered
• Provide the docker deployment as well similar to this https://github.com/topcoder-platform/u-bahn-es-processor#local-deployment-with-docker

Final Submission Guidelines

Please submit the zip file containing the following: 

• Source code for both taas-api and taas-es-processor with Readme with details on how to configure, build, run and test the service
• Submit the verification guide