From Sequential Docker Runners to Parallel ECS: How Marathon Match Scoring Got Faster

If you've competed in a Topcoder Marathon Match, you know the wait. You submit, and then you sit there wondering when the score will land. Behind that wait was an aging piece of infrastructure: a single, hand-configured tester runner that processed submissions one at a time. We've rebuilt it. Here's what changed, why it matters, and what competitors and copilots can expect going forward.

Topcoder Marathon Match scoring has historically depended on a dedicated tester runner: a long-running service, running on a dedicated AWS EC2 instance, that listened for submission events via Kafka, pulled down a submission, ran the tester in a Docker container, parsed the result, and posted review data back into the platform.

That worked, but it was heavy and testing could be slow. The old processor consumed Kafka with `handlerConcurrency: 1` and processed messages with `Promise.each`, so submissions were handled one at a time, in the order in which they were received. Each submission then went through a full Docker lifecycle: build a container, run the tester, wait for completion, collect logs, delete containers/images, upload artifacts, and create review summations.

Configuration was also mostly manual and done by Topcoder staff. A tester bundle carried an execute.sh with the tester class name, solution name, scorecard IDs, time limits, start seeds, and test counts hard-coded per match. Topcoder staff then edited the configuration on the EC2 instance, changed challenge IDs and Docker image names, pulled new images, and restarted runners. The new ECS based flow moves that responsibility into marathon-match-api-v6 (GitHub - topcoder-platform/marathon-match-api-v6). Testers are now versioned API records with source code, class name, compilation status, and compiled JAR output. Creating or versioning a tester triggers asynchronous compilation, and the JAR file is stored in the Topcoder database.

Challenge scoring config is now persisted through the API: scorer selection, scorecard ID, active flag, relative scoring, score direction, timeouts, ECS task definition, and phase mappings for example, provisional, and system scoring.

At runtime, the API consumes incoming submissions from Kafka messages, loads the challenge config, checks the open phase, verifies the tester compiled successfully, checks virus-scan readiness, and launches ECS Fargate scorer tasks. The ECS service injects the challenge ID, submission ID, phase type, seed range, test count, and token into the task, then launches the task.Inside ECS, the Java runner fetches the config, tester JAR, and submission, executes the tester in an isolated runner/scorer environment, uploads artifacts, and posts the scoring callback back to the API. All of the ECS work can happen in parallel for up to 20 submissions at once, allowing system tests and provisional tests to be executed much more quickly.

Additionally, the system tests are now run automatically when the review phase opens.  Previously, multiple EC2 instances were spun up by hand and the system tests were executed semi-manually for the final submissions of each competitor.

The Topcoder platform also now gives copilots a proper setup interface in the Topcoder Work app, meaning that Topcoder staff can be much more hands-off. After a Marathon Match challenge is created, the scorer section lets copilots create a new scorer, publish a new version, watch compilation status, configure phase execution, upload a validation submission, rerun latest submissions, and inspect runner logs.  This was all manual and completed by Topcoder staff in the previous version.

For competitors, the important change is feedback reliability. The tester model remains familiar: submissions are still scored against configured seed ranges for example, provisional, and system phases. But processing is no longer bottlenecked behind one hand-wired Docker runner. Scoring work can run as independent ECS tasks, configuration is stored and validated centrally, tester versions are explicit, and operators have better tools to validate and debug a match before launch.  All of these improvements should lead to much quicker and more reliable scoring and faster challenge completion / system tests.

What this means for you

If you compete, expect scores to come back faster and more predictably, especially during busy system-test phases that used to crawl. If you’re a copilot, you now own the setup end to end without waiting on staff to hand-edit a server. And if you’ve been holding off on Marathon Matches because the scoring felt like a black box, this is a good time to jump back in: the pipeline is more transparent, more parallel, and more reliable than it’s ever been.