Challenge Overview
Giving a picture of human being, there would be various ways to extract a set of coordinates representing his/her joint structure. You will be given a picture of a human being and multiple sets of coordinates representing the joint structure. The goal of this challenge is to design a scoring function to measure how accurate these coordinates are to the given picture and provide reasonable pseudo code (or PoC code) to demonstrate your design.
Please note that this is a data science-based ideation and PoC challenge - there will be a checkpoint you can submit for in order to receive feedback and potentially one of five $100 prizes - we've detailed exactly what we're expecting as outputs below.Bonus Opportunity: $500. The criteria for bonus opportunity is described in Final Submission section.
Background
Recently, the posture recognition technology has been widely used in the video analysis of the surveillance camera. For example, in Gymnastics competitions, one can apply this technique to develop some automatic grading system that can eliminate misjudgment by human judges.In order to realize this, we will need to develop a technology that can match skeleton coordinate data with the target (human athletes) as accurate as possible. The client is currently exploring multiple methods and looking for the community to come up with a new way also.
Are you ready to provide technology and solution that could become the automatic scoring technology and change how we score the Olympic gymnastics competition in the future? We hope to get great solutions from you, Good Luck!
Task Detail
There are already some machine learning models that can predict the human joint coordinates given a still image, which is a frame from a video.
Given the same still image, the video where the still image comes from, as well as a set of predictions (each prediction is a set of joint coordinates), you are asked to design a scoring function, which can measure how accurate these predictions are.
Please provide overall design/algorithm scheme to create a function that can calculate the degree of likelihood of each joint coordinate by comparing to the still image. Please refer to the supplementary materials how this likelihood should be evaluated.
Your algorithm or calculation process idea should take the following 3 data as input (data details provided in Data Description section below);
1. Video: We will be using a Youtube video of a Gymnastics competition as the test case: https://www.youtube.com/watch?v=P09b9Ou0CWU.
2. Still image: a frame from the video containing a gymnastics player in the competition;
3. CSV dataset: A set of extracted joint coordinates of skeletons. Each row is a set of joint coordinates, which are predicted by some machine learning model.
For each extracted joint coordinate in the CSV dataset, your algorithm (or process methods) should output a calculated likelihood score regarding how accurate it is against the still image.
Your algorithm needs to calculate scores of each coordinate within a certain range (higher score should have better accuracy).
Note that just sorting the set of coordinates is not we what we are looking for.
This means the score must be a numerical value (NOT necessarily between 0 and 1) representing the proximity of the CSV dataset compared to the still image.
In addition, the scoring function must meet two criteria, (1) Partial Order Property, and (2) Partial Differentiable Property. For the details of these two criteria, refer to the supplementary materials.
Moreover, your algorithm should be robust enough. It should work for different color brightness or backgrounds. Although having every process automated is preferred, some hand tuning is also acceptable.
The final deliverable is a report explaining the algorithm and pseudocode or PoC that will demonstrate your idea. More details are discussed in Final Submission section.
2. Joint coordinate data set of skeletal: for each still image, there are 200 sets of joint coordinates in the corresponding CSV file.
3. Image superimposed on skeletal images and still images: these pictures are provided for your reference.
4. Program to generate superimposed images. Above (3) was generated from this program using still image and joint coordinates as input.
- - Source code (C++, It can operate in Linux / Windows)
- - Build method is described in the supplementary materials
5. Supplementary material (one PDF).
You can download these materials from this Google Drive link. It's same as the attachments you will see after registration.
Note that the extracted joint coordinates in the provided CSV files are a little far away from the correct positions. In most cases, the distance is at most 20cm. In general, one can assume the distances of the extracted joint coordinates from the correct positions are within 10cm.
2. Abstract / Description : High level overview / statement of your idea
- - Outline of the approach of the algorithm implementation
- - Outline of each functional unit of the algorithm implementation
- - Summarized conclusion
3. Details :
- - Detailed description. You must provide details of each function unit and details of how it should be implemented
- - - - Description of the entire mechanism
- - - - The advantage of your idea - why it could be better than others
- - - - If your idea includes some theory or known papers;
- - - - - - Reason why you chose
- - - - - - Details on how it will be used
- - - - - - Reference to the papers of the theory
- - - - Reasonings behind the feasibility of your idea
- - Resolution of the following two conditions related to the likelihood function
- - - - Partial Order Property
- - - - Partial Differentiable Property
- - Pseudo code(or PoC code), and its description
4. Appendix(optional) :
- - Bibliography, A reference to the paper, etc.
Checkpoint Submission - Five Prizes of $100
In this challenge, we allow checkpoint submissions. In the checkpoint submission, you must at least include the “Abstract / Description” part.
In order to be eligible for a checkpoint prize you must submit to this challenge by Feb 18th at 10:00 PM EST.
Any submissions timestamped before then will be eligible and considered in the running for a prize for both the checkpoint and final submission. After this time you will only be eligible for final submission prizes.
The advantages of submitting to the checkpoint phase will be to get client feedback on your ideas and the opportunity to receive one of the five $100 checkpoint prizes.
You are not required to submit for the checkpoint in order to submit for the final phase.
Final Submission
In the final submission, you must submit all items described in Contents section above.
** Bonus Opportunity: $500 **
If we see something truly high quality and the solution can be implemented straightforwardly from your idea, we reserve the right to give out an additional $500 bonus prize to the submission.
1. Feasibility of the Idea (Validatable possible implementation methods have been specified for the particular item of the following) (60 %)
- - Partial Order Property
- - Partial Differentiable Property
2. Ease of implementation (40 %)
- - That pseudo code or PoC code is presented
- - The above code Implementation is easier than other submissions
For example, following will be judged higher
1. Having PoC code is better than just pseudo code, but if pseudo code report has a better solution and highly feasible reasonings to accurately calculate the scores, the latter report will be evaluated better.
Note: We will be looking mainly at “Abstract / Description” for the checkpoint submissions.
Supplementary materials
Please use the following link to retrieve the most updated version.
https://docs.google.com/presentation/d/1PyJEVIjKFObfxyiYAx-XYQKDSY0T_xRCeUCxHed4G84/edit?usp=sharing
This means the score must be a numerical value (NOT necessarily between 0 and 1) representing the proximity of the CSV dataset compared to the still image.
In addition, the scoring function must meet two criteria, (1) Partial Order Property, and (2) Partial Differentiable Property. For the details of these two criteria, refer to the supplementary materials.
Moreover, your algorithm should be robust enough. It should work for different color brightness or backgrounds. Although having every process automated is preferred, some hand tuning is also acceptable.
The final deliverable is a report explaining the algorithm and pseudocode or PoC that will demonstrate your idea. More details are discussed in Final Submission section.
Data Description
1. Still image: there are 3 still images taken from video recordings.2. Joint coordinate data set of skeletal: for each still image, there are 200 sets of joint coordinates in the corresponding CSV file.
3. Image superimposed on skeletal images and still images: these pictures are provided for your reference.
4. Program to generate superimposed images. Above (3) was generated from this program using still image and joint coordinates as input.
- - Source code (C++, It can operate in Linux / Windows)
- - Build method is described in the supplementary materials
5. Supplementary material (one PDF).
You can download these materials from this Google Drive link. It's same as the attachments you will see after registration.
Note that the extracted joint coordinates in the provided CSV files are a little far away from the correct positions. In most cases, the distance is at most 20cm. In general, one can assume the distances of the extracted joint coordinates from the correct positions are within 10cm.
Submission
Commercialization Requirements
Note that an important goal of this project is to build a commercial software. If you want to use some open-source software that can not be used for commercial purposes, please also discuss the effect in your submission and show the development potential of alternative code.
Contents
The final submission should be a report, more like a technical paper. It should include, but not limited to, the following contents. The client will judge the feasibility and the quality of your proposed likelihood function.
2. Abstract / Description : High level overview / statement of your idea
- - Outline of the approach of the algorithm implementation
- - Outline of each functional unit of the algorithm implementation
- - Summarized conclusion
3. Details :
- - Detailed description. You must provide details of each function unit and details of how it should be implemented
- - - - Description of the entire mechanism
- - - - The advantage of your idea - why it could be better than others
- - - - If your idea includes some theory or known papers;
- - - - - - Reason why you chose
- - - - - - Details on how it will be used
- - - - - - Reference to the papers of the theory
- - - - Reasonings behind the feasibility of your idea
- - Resolution of the following two conditions related to the likelihood function
- - - - Partial Order Property
- - - - Partial Differentiable Property
- - Pseudo code(or PoC code), and its description
4. Appendix(optional) :
- - Bibliography, A reference to the paper, etc.
Format
1. A document should be minimum of 2 pages in PDF / Word format to describe your ideas.
2. It should be written in English.
3. Leveraging charts, diagrams, and tables to explain your ideas is encouraged from a comprehensive perspective.
Requirement To Win a Prize
Checkpoint Submission - Five Prizes of $100In this challenge, we allow checkpoint submissions. In the checkpoint submission, you must at least include the “Abstract / Description” part.
In order to be eligible for a checkpoint prize you must submit to this challenge by Feb 18th at 10:00 PM EST.
Any submissions timestamped before then will be eligible and considered in the running for a prize for both the checkpoint and final submission. After this time you will only be eligible for final submission prizes.
The advantages of submitting to the checkpoint phase will be to get client feedback on your ideas and the opportunity to receive one of the five $100 checkpoint prizes.
You are not required to submit for the checkpoint in order to submit for the final phase.
Final Submission
In the final submission, you must submit all items described in Contents section above.
** Bonus Opportunity: $500 **
If we see something truly high quality and the solution can be implemented straightforwardly from your idea, we reserve the right to give out an additional $500 bonus prize to the submission.
Judging Criteria
You will be judged on the quality of your ideas, the quality of your description of the ideas, and how much benefit it can provide to the client. The winner will be chosen by the most logical and convincing reasoning as to how and why the idea presented will meet the objective. Note that, this contest will be judged subjectively by the client and Topcoder. However, the judging criteria will largely be the basis for the judgment.1. Feasibility of the Idea (Validatable possible implementation methods have been specified for the particular item of the following) (60 %)
- - Partial Order Property
- - Partial Differentiable Property
2. Ease of implementation (40 %)
- - That pseudo code or PoC code is presented
- - The above code Implementation is easier than other submissions
For example, following will be judged higher
1. Having PoC code is better than just pseudo code, but if pseudo code report has a better solution and highly feasible reasonings to accurately calculate the scores, the latter report will be evaluated better.
Note: We will be looking mainly at “Abstract / Description” for the checkpoint submissions.
Supplementary materials
Please use the following link to retrieve the most updated version.
https://docs.google.com/presentation/d/1PyJEVIjKFObfxyiYAx-XYQKDSY0T_xRCeUCxHed4G84/edit?usp=sharing
Final Submission Guidelines
You can submit at most TWO solutions but we encourage you to include your great solution and details as much as possible in a single submission.