Topcoder - Marathon Match New Scoring System - Initial PoC

Welcome to Topcoder - Marathon Match New Scoring System -  Initial PoC.

Challenge Objectives

 

• For this contest, we are looking at trying some new technologies to assembly the new scoring system for Marathon Match challenges.
• For that we would like to experiment the CLING https://github.com/root-project/cling

Project Background

 

• Topcoder is updating all its submission and review system to a new plataform.
• One of the systems that needs to be migrated is the Marathon Match Scoring System.
• The Marathon Match Scoring System is responsible for retrieving the member submission in different coding languages and running the tests for a given problem provided in the challenge and return the submission score.

Technology Stack

 

• C++

Individual requirements
 

1. We would like to create a library using CLING to allow running code against arbitrary data and source code for usage during Marathon Match challenges. References:

https://github.com/root-project/cling

https://cdn.rawgit.com/root-project/cling/master/www/index.html

2. Today in Marathon Match member submits C++ code with a given signature to run test data and produce the results. Based on the results a given score is produced. Example:

- Problem statement: https://community.topcoder.com/longcontest/?module=ViewProblemStatement&rd=10747&pm=7604 

- Submissions: http://community.topcoder.com/longcontest/stats/?module=ViewSystemTestResults&rd=10747&pm=7604

3. The library should allow running the solution submitted by member for a challenge problem statement. Every single problem will have its own method signature so the library must be flexible to handle this.

4. To allow a great amount of flexibility JSON must be used as input data format, provide method signature and return the results. See below for class definition.

name:

- Runner

constructor:

- Runner(json signature)

public methods:

- json runSubmission(json data, std::string &submissionCode)

signature will hold the defintion of the public method signature that the submitted code should expose.

data will be any arbitrary json array. Each entry of the array must match the provided input parameters for public method. 

submissionCode is a string that holds the code submitted by user, which should define the class and public method defined in signature.

In the above provided statement we have the following definition:

Class: LumberjackExam
Method: examineForest
Parameters: int, int, int, double
Returns: int
Method signature:int examineForest(int height, int width, int time, double density)

For the above example data would be:

[
    {
        "input": [ 10, 10, 10, 0.5 ]
    }
    ...
]

For the above example signature would be:

{
  "input": [ "int", "int", "int", "double" ],
  "output": "int",
  "className": "LumberjackExam",
  "method": "examineForest"
}

For the above example results would be:

{
    "error": null, // compile errors,
    "statistics": {
         "time": { "mean": 1020, "max": 1100, "min": 900 },
         "memory": { "mean": 1020, "max": 1100, "min": 900 }
    },
    "results": [
        {
            "error": null, // running error of input value 1
            "statistics": {
                "time": 1000,  // (ms), running time of input value 1
                "memory": 1000, // Kbytes, memory usage of input value 1
             },
             "output": 20 // output of input value 1
        },
        {
            "error": "out of memory", // running error of input value 1
            "statistics": {
                "time": 1000,  // (ms), running time of input value 1
                "memory": 1000, // Kbytes, memory usage of input value 1
             },
             "output": null // output of input value 1
        },
        ...
    ]
}

6. The runSubmission will use CLING to run the submission source code using the input data and produce the results.

7. Accepted input and output parameters types are int, double, std::string, vector<int>, vector<double>, vector<std::string>

8. Could should compile in Linux and Mac OS. For linux use g++ 7+. Standard C++11 must be used.

9. Create a test program to verify the application for different signatures and source codes (must cover all the types in item 7).
 

Deployment guide and validation document

You will need to create a deployment guide (README.md) and validation document (Validation.md).

The README.md created must show how to:

• Configure and Build application

The Validation.md created must show all the steps necessary to verify the application:

• Verify the code for different test cases