TCO19 India Regional Event Development - SRM Problem Archive Elasticsearch

 

• Target environment: Elastic Cloud (Elasticsearch, Kibana)

• Basic Requirements: Using Elasticsearch, Kibana to index and search SRM Problem Archive

Project Background

Topcoder has amassed a great wealth of SRM problems since it was first founded in 2001. They are great training, learning and educational content. In this challenge, we are seeking your help to leverage the powerful Elastic Cloud to

1. Upload the problem archive to the Hosted Elasticsearch on Elastic Cloud

2. Search, filter or visualize the problems in the Host Kibana on Elastic Cloud.

Technology Stack

• Elastic Cloud

• Elasticsearch

• Kibana

Individual requirements

Notes: We already have a GitHub project to show a Search UI + Elasticsearch Example, See https://github.com/elastic/search-ui/tree/master/examples/elasticsearch, you can work based on this GitHub repo. That’s an optional requirement.

 

The problem archive contains 4693 problems, we provided two CSV files that contain all the required fields. Here are the descriptions of each file and fields.

• Problem_Description_Temp.csv, the fields are

  • problem_id: The problem ID. - Unique ID of the Problem

  • method_name: The required method name. - Method name

  • class_name: The required class name. - Generally the problem name, technically the class name which member has to implement for writing the solution.

  • problem_text: The problem text or the problem description.

• ProblemArchive.csv, the fields are

  • problem_name: The name of the problem. - This is similar to the class name is the Problem_Description_Temp.csv

  • contest_name: The name of the contest can be SRM, or TCO18, TCO17 or TCCC(collegiate contest)

  • contest_date: The date of the contest

  • problem_id: The problem ID - Unique ID of the Problem

There are three levels of problems in each division. (1, 2 and 3. 1 stands for easy, 2 stands for medium, and 3 stands for hard)

• div1_level: The level of Division 1.

• div2_level: The level of Division 2

• div1_success: Success rate of the Div I problem

• div2_success: Success rate of the Div II problem

• round_id:  The round id.

• Categories: Problem category, can be:

  • Advanced Math

  • Brute Force

  • Dynamic Programming

  • Encryption/Compression

  • Geometry

  • Graph Theory

  • Greedy

  • Math

  • Recursion

  • Search

  • Simple Math

  • Simple Search, Iteration

  • Simulation

  • Sorting

  • String Manipulation

  • String Parsing

• writer_handle: Username/handle of the writer

• writer_id: user id of the writer

 

1. Data Preparation

Please note the data in ProblemArchive.csv are not clean, the problem_text field might contain a newline, comma, double quotes characters that might clutter the CSV format. You might need to clean up the ProblemArchive.csv to correct the CSV format before uploading it to the Elastic Cloud.

Login to the Elastic Cloud with your account. If you don’t have an account yet, you can start a free trial here.

After login, create a deployment, then you will have a cluster with ElasticSearch and Kibana instances. Then you can import the two CSV files into the ElasticSearch.

Here is the tutorial of how to upload the CSV files into ElasticSearch.

2. Configure Kibana and ElasticSearch.

We want you to configure the Kibana & ElasticSearch for filters, here are some sample use cases of filters.

1. Filter problems by writers.

2. Search problem by some keywords or sentence in the problem description.

3. Filter problems by categories

4. Filter problems on difficulties

5. Filter problems on the number of submissions made to it

6. Filter problem on which user who wrote the problems(like I want to see problems written by espr1t)

 

Also, Kibana provides a Dashboard to visualize data, here are some sample use cases of Dashboard.

1. How problem trends and maybe trends of search data

2. Display the most viewed problem or difficulty wise problem distribution

3. Success rate wise problem distribution

4. etc...

 

Please note you are more welcome to create more filters and Dashboard pages to showcase more useful and interesting use cases. The filter and Dashboard use cases are not limited to what listed above. We want you to be innovative and make the problem archive interesting

.

Tips: Here are the video tutorials to help you get started in Elastic Cloud.

Final Review & Scoring Guidelines

Your submission will be reviewed subjectively, the review scoring guidelines are

 

(1) Completeness of your submission (50%)

• Did you finish the Elastic Cloud setup?

• Did you import the CSV data into ElasticSearch successfully?

• Did you finish at least 1 filter use case by setting up Kibana?

(2) Filters and Dashboard Showcase (30%)

• Did you finish all the listed filter use cases?

• Did you finish all the sample use cases of Dashboard?

(3) Innovation (20%)

• How much benefit are your new filters?

• Is it interesting what your Dashboard demonstrated?

• Are your submission and idea innovative overall?

• [Optional] Use this Search UI Example to add a UI to your search engine.

  • CSS/Design is not mandatory

  • Need to modify the example to fit your dataset.

Final Submission Guidelines

Please zip all the following content in a zip archive.

1. A Kibana link of your Elastic Cloud, you also need to provide the credentials for login, you can also create a login for the reviewer and provide that. The reviewer will log in to your Kibana instance and review your submission.

2. Simple documentation to describe how to use your Kibana to apply the filters and view the Dashboard.

3. A demo video to showcase the filters and Dashboard (Not mandatory but might be helpful for the reviewer to review)