Challenge Overview
Challenge Objectives
-
Develop frontend&backend for a POC document search app
-
Integrate with Watson Discovery API
Project Background
-
Our client, an insurance company wants us to build a system that will crawl internal and external data sources and provide search results for the relevant claims.
-
In this challenge, we’ll build configure Watson discovery service to crawl external data sources, build a UI for the end user to enter queries and view results, and build a backend that will just proxy the query requests to the discovery service.
-
Future challenges will implement further query filtering options and add more external data sources.
Background
Our client has a lot of large payouts in $250k-$2.5MM where a person deciding on a claim (adjustor) makes all the decisions without oversight and governance in place. Their job consists of analyzing the policy and the claim and then manually researching current historical precedent and current trends/news around industry and litigation of these claims.
Today, their best source of research is google.com or a paid for research forum – and the adjustor has to take the initiative to perform the research vs. being pushed content. This is a blend of structured and unstructured data (i.e. take structured claims data and then crawl over public and trusted sources to bring together research / opinion of unstructured data) to help guide the claims adjustor’s strategy of settle the claim, litigation or mediation, reinsurance, and or closing the loop of communication with underwriting about how to go to market with future policy language.
Technology Stack
-
Watson Discovery Service
-
NodeJS
-
Angular / Angular Material
Code access
We’re starting a new codebase for, so you should create the project structure.
Winning submission of the ideation challenge is available in the forums - it contains the high level overview of what we’re trying to build in this project.
Individual requirements
-
Configure Discovery service to crawl the external data sources
Create a Watson environment and configure at least 3 external data sources to be crawled and indexed (pick the data sources from the list provided in the forums) and the default news data source. Each data source should be configured as a separate collection (ex legal, news, insurance)
NOTE: Free tier of the Watson service allows for 1000 documents at a time so limit the number of crawled documents
-
Build the application backend
App backend will be really simple - with one endpoint that will accept the search query from the frontend and send the query to the Discovery service API to query multiple collections (list of collections to query should be provided as an input to the endpoint, see the frontend requirements below). All credentials should be configured using environment variables. No authentication is required at this time.
-
Application frontend
Frontend will be a single page app with the following options
-
Enter search query & configure filters
-
View results - documents, entities, keywords, categories
Here is a rough wireframe of the required elements
User will enter the query and select the desired collections to search. Loading spinner should be shown while loading the search results. Search results section should display returned entities, keywords and categories and the actual documents list with pagination. Passage search should be used for displaying the results snippet (highlighting the relevant section of the snippet), and natural language for matching the relevant documents.
Create a README file with details on how to deploy and verify the frontend/backend and Discovery service configuration. Deploy a sample app to Heroku. Unit testing is out of scope.
What To Submit
Submit the full source code
Submit the build/deployment documentation
Submit a short demo video