Challenge Overview
App environment:
- Nodejs
- Kafka
- ElasticSearch
- Docker
- Docker Compose
Basic requirements for this challenge:
- Update Topcoder Member Processor ES to follow the expected structure. And clean up unused code.
- Update Topcoder Member Service to to follow the expected structure.
Base Submissions:
https://github.com/appirio-tech/ap-member-microservice
https://github.com/topcoder-platform/member-processor-es
Deployment environment requirements
- localhost
- livesite (AWS)
Notes from current implementation
- We need you update existing nodejs app that linked with Kafka and ElasticSearch.
- Right now, our ap-member-microservice will send message to BusEvent to linked with member-processor-es
- For this challenge we need you update both repo to follow the expect structures.
- Current app will listen to these following topic on kafka:
-- member.action.profile.create
-- member.action.profile.update
-- member.action.profile.trait.create
-- member.action.profile.trait.update
-- member.action.profile.photo.create
-- member.action.profile.photo.update
-- member.action.profile.delete
-- member.action.profile.trait.delete
- When a message arrives on either of the above topics, the processor will parse the payload and create / update / delete data in Elasticsearch
- So, taking an example, the processor will first receive a message in member.action.profile.create type to create data in ElasticSearch.
- The processor will then subsequently receive the update and delete messages for the data, which it will update accordingly in ElasticSearch.
- Pay attention each data format are different
- The data to create / update in ElasticSearch will be present in the message received in Kafka itself.
- You can check if message delivered for each topics here: https://lauscher.topcoder-dev.com/. Login with: TonyJ/appirio123
Submission Requirements
1). Fix Member Profile ES Structure
Download current and expected structure JSON files here: https://drive.google.com/open?id=1OZukEVv-t-rv7NCpA-TZEMVi23z9kTcr
01_current_ES_Search_Response.json
- This is current JSON structure when pull all member profile data from single user
- You can see demo url here: https://search-topcoder-squ62azmqlwkvnmztjmk4cq5fq.us-east-1.es.amazonaws.com/members5/_search?q=userId:40154303
02_expected_ES_Search_Response.json
- This is expected JSON structure when pull all member profile data from single user
Current Response
- Current id for each topics merged from profile+id, for example “profile40154303”, “profile40154303traitdevice”, “profile40154303traitcommunities” and so on.
- Each response for single user separated on same level, we need them to follow required structure below
- There are some resource for profile, trait and photo. Did we need keep that with the updated structure? Looking for your suggestion.
Expected Response
- User id information need just use user id
- Response Structure need always displayed user profile information
- Response need capture user profile, traits and photo in 1 node tree.
- All traits need placed under traits level
- For each trait need use traitId, for example: “basic_info”, “communities”, “customer_info” etc
- createdAt, createdBy, updatedAt need represent for profile, traits or photo
- Follow this expected response as close as possible.
- You might need update the ap-member-microservice, to make sure it send correct message structure.
- Anything else we need aware? Lets discuss if you need some clarifications.
2). Update Unit Tests
- Update unit test as your edits required above
- Clean up unused code
- Submit a git patch that implement the tasks requirements on both required repos:
https://github.com/appirio-tech/ap-member-microservice
https://github.com/topcoder-platform/member-processor-es
- Updated Documentation (if Any)
- Updated Postman Collection (if Any)
- Verification document contains steps to verify your solution. (if Any)
- Updated Swagger file
- Nodejs
- Kafka
- ElasticSearch
- Docker
- Docker Compose
Basic requirements for this challenge:
- Update Topcoder Member Processor ES to follow the expected structure. And clean up unused code.
- Update Topcoder Member Service to to follow the expected structure.
Base Submissions:
https://github.com/appirio-tech/ap-member-microservice
https://github.com/topcoder-platform/member-processor-es
Deployment environment requirements
- localhost
- livesite (AWS)
Notes from current implementation
- We need you update existing nodejs app that linked with Kafka and ElasticSearch.
- Right now, our ap-member-microservice will send message to BusEvent to linked with member-processor-es
- For this challenge we need you update both repo to follow the expect structures.
- Current app will listen to these following topic on kafka:
-- member.action.profile.create
-- member.action.profile.update
-- member.action.profile.trait.create
-- member.action.profile.trait.update
-- member.action.profile.photo.create
-- member.action.profile.photo.update
-- member.action.profile.delete
-- member.action.profile.trait.delete
- When a message arrives on either of the above topics, the processor will parse the payload and create / update / delete data in Elasticsearch
- So, taking an example, the processor will first receive a message in member.action.profile.create type to create data in ElasticSearch.
- The processor will then subsequently receive the update and delete messages for the data, which it will update accordingly in ElasticSearch.
- Pay attention each data format are different
- The data to create / update in ElasticSearch will be present in the message received in Kafka itself.
- You can check if message delivered for each topics here: https://lauscher.topcoder-dev.com/. Login with: TonyJ/appirio123
Submission Requirements
1). Fix Member Profile ES Structure
Download current and expected structure JSON files here: https://drive.google.com/open?id=1OZukEVv-t-rv7NCpA-TZEMVi23z9kTcr
01_current_ES_Search_Response.json
- This is current JSON structure when pull all member profile data from single user
- You can see demo url here: https://search-topcoder-squ62azmqlwkvnmztjmk4cq5fq.us-east-1.es.amazonaws.com/members5/_search?q=userId:40154303
02_expected_ES_Search_Response.json
- This is expected JSON structure when pull all member profile data from single user
Current Response
- Current id for each topics merged from profile+id, for example “profile40154303”, “profile40154303traitdevice”, “profile40154303traitcommunities” and so on.
- Each response for single user separated on same level, we need them to follow required structure below
- There are some resource for profile, trait and photo. Did we need keep that with the updated structure? Looking for your suggestion.
Expected Response
- User id information need just use user id
- Response Structure need always displayed user profile information
- Response need capture user profile, traits and photo in 1 node tree.
- All traits need placed under traits level
- For each trait need use traitId, for example: “basic_info”, “communities”, “customer_info” etc
- createdAt, createdBy, updatedAt need represent for profile, traits or photo
- Follow this expected response as close as possible.
- You might need update the ap-member-microservice, to make sure it send correct message structure.
- Anything else we need aware? Lets discuss if you need some clarifications.
2). Update Unit Tests
- Update unit test as your edits required above
- Clean up unused code
Final Submission Guidelines
Challenge Deliverables- Submit a git patch that implement the tasks requirements on both required repos:
https://github.com/appirio-tech/ap-member-microservice
https://github.com/topcoder-platform/member-processor-es
- Updated Documentation (if Any)
- Updated Postman Collection (if Any)
- Verification document contains steps to verify your solution. (if Any)
- Updated Swagger file