Challenge Overview
The goal of this challenge is to build a set of scripts that cache a local copy of external APIs, and build set of APIs to expose these cached data.
What you are addressing in this challenge :
- Script to cache O*Net data in mongodb (caching layer)
- API endpoints to pull data from the caching layer
- API endpoints to handle assessment flow
Challenge Requirements
UI Prototype
https://brilliant-careers-static.herokuapp.com/
We are addressing the Assessment and Career Exploration pages in this challenge.
Existing API
The source code is provided in challenge forums.
External API
We are integrating with ONet API.
O*Net
API Documentation:
https://services.onetcenter.org/reference/
Endpoint:
https://services.onetcenter.org/ws/
Terms of Service:
https://services.onetcenter.org/terms
Caching Scripts and Services
As there are limits to access these external api, we need to build a cache layer that copy the data we need to serve in the website, and have a script to regularly updating this cached version.
What we need to address this requirement:
- A script for each external API that can regularly run to pull data and update existing data (it is important that script works properly with existing data update)
- The script will use Mongodb. We will flatten the pulled data into collections as much as possible instead of maintaining same structure from the external APIs.
- The services to expose access these cached data should be decoupled from the API we will build in the challenge, the reason that in future we might switch the mongodb to use a memcache or redis caching services or move to indexing service such as apache solr, we want to make this transition without refactoring any API code, we only update the services, input/output should remain the same. So you need to add another level beneath the service you will write to talk to this caching service.
O*Net Integration
Assessment Page
Reference: https://brilliant-careers-static.herokuapp.com/Assessment.html
- This page takes user to O*Net assessment flow, it is a 60 questions form.
- The questions are pulled from https://services.onetcenter.org/reference/mnm#ip_questions
- The questions and possible answers should be stored in a single collection (assessment_questions) in DB.
- This mongo collection will be used to pull questions every time user plays the assessment flow.
- This mongodb collection will always have additional fields such as image and any other fields that needed but not present in the O*Net questions information.
Career Exploration
Reference: https://brilliant-careers-static.herokuapp.com/Career_Exploration.html
- In this page we want to address the recommended careers areas, including filtering/searching which is not supported by ONet API.
- The recommended careers will fall into 4 categories only : match, technology, science, and engineering.
- The caching script should pull all careers that can be found in O*Net for each category of the 4 categories.
- To pull careers : https://services.onetcenter.org/reference/mnm#search by passing the category as keyword.
- For each response there will be pagination links to iterate through all careers to pull.
- Each career you need to use the api URL in the response to pull the career information, then recursively access all API URLs in the career xml response and cache them as well excluding check_out_my_state and explore_more.
- The db collection can be single one where each document contains all the information pulled about the career including the nested URLs pulled content. Again information should be flattened as much as possible.
API Endpoints
You will build the following endpoints in this challenge :
- Assessment Leaderboard
- GET /assessment/leaderboard
- Used in this page https://brilliant-careers-static.herokuapp.com/Assessment.html (bottom right)
- This is public endpoint.
- Logic :
- Retrieve users who completed the assessment using points in descending order with pagination enforced.
- Pagination parameters are input, if not provided, default values should be use.
- Sum the assessment_answer points, grouped by user_id, and ordered using aggregated points in descending order.
- The response should include the pagination parameters used, and the total matching value without pagination.
- Start Assessment
- POST /assessment
- Used in this page https://brilliant-careers-static.herokuapp.com/Assessment_Test.html
- This is protected endpoint.
- Logic
- We call this endpoint to track user starting to work on the assessment.
- This will add new entry to user_activites of type ‘start_assessment’, it set date as current date, and object should store a counter attribute to indicate the number of tries.
- If user already have an active assessment, then return it instead of creating new one.
- Submit Assessment answers
- PUT /assessment/answers
- Used when user finish the assessment questions.
- Logic
- Check if user has active assessment, otherwise, return error.
- Store the answers in postgres DB assessment answers table.
- User can redo assessment, so we need to keep track of retries in the history, so the assessment answer table might have multiple records for same user.
- The table should store answers in single field concatenated.
- Assign user the assessment module badge (should be configurable) if not already assigned.
- Calculate answers using O*Net answers api https://services.onetcenter.org/ws/mnm/interestprofiler/results?answers=
- Store the response in assessment answer table, in a field called ‘results’
- Sum the scores user get from results, and store it in UserActivity with type ‘assessment_completed’.
- Return the badge, and score to be displayed here https://brilliant-careers-static.herokuapp.com/Assessment_Test.html
- Cancel Assessment
- DELETE /assessment
- Used in this page https://brilliant-careers-static.herokuapp.com/Assessment_Test.html
- User can cancel the assessment, so we need to cancel tracking it in backend.
- Logic
- Check if user has active assessment.
- Insert new record ‘assessment_cancelled’ and use same as the corresponding ‘start_assessment’ counter attribute value.
- Return error if no assessment active to cancel.
- Get Assessment Questions
- GET /assessment/questions
- This is public endpoint
- Logic
- Return the 60 questions from cached O*Net service, and possible answers.
- Get Assessment Results
- GET /assessment/results
- This is protected endpoint because we need user identity to pull user results.
- Logic
- Pull the latest assessment answer record and return the results attribute.
- Get Recommended Careers
- Get /careers/recommended/[topic-slug]?filter=<filter-parameters>&search=<search-keyword>
- Will be used at this page (at bottom) https://brilliant-careers-static.herokuapp.com/Career_Exploration.html
- Public endpoint
- The [topic-slug] will be one of the four categories : math, technology, science, and engineering. It is optional parameter.
- Logic
- Pull a set of recommended careers using the passed slug with pagination enforced.
- If slug not passed then we pull random careers data.
- If filter parameters are passed or search keyword, it should be applied to the search. The mongo DB should enable the text indexes so we search the full DB.
- The data needed to be returned
- Career image
- Career title
- Career median annual salary
- The ID to link with ‘learn more’ section
- Pagination information (size, offset, and total)
- Get Skills
- GET /careers/skills
- This is public endpoint to be used in the filter drop down in the filter section in Career Exploration page
- Logic
- This is a lookup table, you can reuse the existing one.
- This lookup table should be populated during caching O*Net data to put unique (no duplicate) skills.
- Get Career Details
- Get /career/<career-id>
- Public endpoint
- This endpoint will be used to populate the following areas :
- The top area in this page https://marvelapp.com/10hc7ec#14071106
- Essential Functions and Responsibilities https://marvelapp.com/10hc7ec#14071107
- What skills do I need for the job https://marvelapp.com/10hc7ec#14071108
- Logic
- Pull the needed fields from cached O*Net data.
- If anything not clear about specific field/section please ask in forums, you can simply map UI fields with the O*Net data by looking at the responses from the O*Net API.
- Get Similar Careers
- GET /careers/similar
- Public endpoint
- Logic
- Using the career associated slug pull similar careers with pagination parameters, if no pagination parameters, then default values should be used.
- Favorite a Career
- POST /career/<id>/favorite
- This is protected endpoint
- Will be used to Save career to Profile option in career details page.
- Logic
- Add new table to store user favorited careers
- Get Favorited Careers
- GET /careers/favorite
- This is protected endpoint
- Logic
- Using pagination parameters, return the user favorite careers
- If pagination missing then use default value
- If no favorite then return empty array
Notes
- Prefix for api routes : /api/
- Gives command to run the scripts as cron jobs to verify the cache update works without any problems.
- All api endpoints input parameters should be properly validated.
- Error messages should be in json format, with fields : message, error code, and error flag (true).
- Postman file should be updated with proper endpoints and test cases.
- Proper logging should be added for all endpoints.
- All code must be linted using eslint.com
- All code must be properly documented.
Technology Stack
- NodeJS 4.x
- Postgres 9.4
- Heroku
Final Submission Guidelines
Deliverables
- Git patch file that include all changes.
- Deployment guide should be the README filed updated with all required steps to deploy and test your app locally and on heroku.
- Verification document.