Challenge Overview
Challenge Objectives
The objective of this challenge is to train a machine learning or deep learning model (and wrap it into a function and a CLI) that can classify shoe images by type.
The primary requirements are:
-
Train a ML/DL model to classify images of shoes: The ML/DL model should be able to take as input an image and classify them in one of the following categories: sneakers, boots, loafers, sandals, flip_flops, soccer_shoes, no_shoe_found. A Jupyter notebook of the training steps needs to be shared in the submission.
-
Assemble your own dataset: The contestants are free to assemble their own image dataset to train their models. (Note that while acquiring the images or building the dataset, no licensing agreements should be violated such that it does not allow commercial use of the final trained model).
-
The model should be wrapped into a function named classify_images: A function called classify_images should be created, which should be able to receive a set of images as input and should return their categories as output.
Project Background
The goal of the project is to build a demo image classification application for our client EdgeNet that can take as input an image, classify it and then perform some additional operations based on the classification.
Note that in this particular challenge, we are NOT building the application or the API, and will also NOT build the functionality in this challenge to perform the additional operations based on the classification. These requirements will be covered in the next challenge.
In the current challenge, the tasks are limited to building the classification model and wrapping it into a function, such that it is very easy to invoke this function by the additional API and web components that will be built in the future.
Challenge Details
-
Train a ML/DL model to classify images of shoes: The ML/DL model should be able to take as input an image and classify them in one of the following categories: sneakers, boots, loafers, sandals, flip_flops, soccer_shoes, no_shoe_found.
Here, to understand what each label means just perform an image search by converting the underscore-delineated labels mentioned here into space-delineated terms. For example, soccer_shoes means 'soccer shoes', flip_flops means 'flip flops' and no_shoe_found means 'no shoe found'.
Note that the output label NoShoeFound will be inferred as 'the image does not contain any shoe of the type sneakers, boots, loafers, sandals, flip flops or soccer shoes. -
Assemble your own dataset: The contestants are free to assemble their own image dataset to train and validate their models. (Note that while acquiring the images or building the dataset, no licensing agreements should be violated such that it does not allow commercial use of the final trained model).
Here, you can search for images using the category terms as discussed above, and download individual images or the dataset (without violating any commercial-use requirements of the final model & script.) The mentioned types of shoes are pretty common, and in case the contestant is not familiar with them, it will be the responsibility of the contestant to familiarize themselves with each type of shoes and select the correct images while assembling their dataset.
Note - the image can contain either one shoe or both shoes, and it should not necessarily be centered.
Important Note on size limit of the dataset - To ensure that individual with comparatively more computational or financial resources don't have an unfair advantage in the contest, the maximum number of training images for each category is limited to a maximum of 400 images. For instance, your model should not be trained on more than 400 images for any particular category.
Furthermore, the validation should be performed on a maximum of 150 images of each category. The total training dataset should not contain more than 2800 images, and validation set should not contain more than 1050 images.
Jupyter Notebook - The contestants are required to train their models in a Jupyter notebook and share that notebook in the submission. All necessary training instructions should be added in the notebook such that the training can be repeated if required. -
The model should be wrapped into a function named classify_images: A function called classify_images should be created, which should be able to receive a set of images as input and should return their categories as output. This function will wrapped into an API in the subsequent challenges.
Note - classify_images(list_of_N_image_paths) -> list_of_N_classes. Here the path can be either links to the images or path if it is a local file. Note that in the subsequent challenges we might need to modify list of image links to something else like list of multiple Base 64 encoded JSON payloads, hence it is expected that the function and the script in general is well-commented and hence easy to modify by subsequent developers.
Optional Prize - If any submission implements an API and manage to win the 1st or 2nd prize, they will be paid an additional $100 separately. Note that during review no points will be deducted if API has not been implemented. -
A Jupyter Notebook to test the model - The submission should include a notebook where it's possible to add a test set and get as output a csv output of the scores. Each row of the csv should contain the name of image and its corresponding class. To be exact the format should be: <name_of_image_file>,<predicted_label>. The testing set will be in the form of a folder containing 7 sub-folders (one sub-folder for each class) containing images of that class. Note if there is any difficulty in understanding the format of the test set, please refer to the bear dataset structure in FastAI lecture 3 of their 2020 Course.
Important Note - The Jupyter Notebooks are required only for the training and testing part. The classify_images function and its related script should be in the form of actual Python .py extension files, which can be executed in the command line.
Learning Resources
For contestants looking for some resourced to learn or brush up their skills that are relevant for this challenge, a great place would be to refer to the the FastAI courseof their 2020 Course, particularly Lecture 3.
Review Criteria
The review will be performed in the form of a combination of subjective and objective review criteria:
-
In the objective review, the model's Micro F1 Score will be calculated using the testing Jupyter Notebook as mentioned above.
-
In the subjective review, the code quality, function structure, documentation and the overall submission will be subjectively reviewed to judge how fit it is for subsequent development work. Here, the review will be almost similar to the standard Code challenge review.
-
The weighage would be as follow:
- Objective Review - 80%
- Subjective Review - 20%
Technology Stack
-
Deployment Environment - Deployable on any platform that can execute Python via a terminal or command line
-
Language - Python 3.9.5 (mandatory)
What To Submit
-
Source code - The source code of the script + two Jupyter Notebooks (one of training & validation, one for testing the model)
-
Documentation - Instructions to deploy the solution from scratch.
-
Verification Instructions - The documentation and the Jupyter notebook should contain detailed instructions.
Video Bonus: -
Contestants can optionally also include a video demonstrating their script and submission meets all the requirements. The video can be uploaded to YouTube, Google Drive, Dropbox or elsewhere and the link can be included in the submission archive.
-
The video should demonstrate the Jupyter notebooks and script in action. If the submission manages to win a placement (i.e. a top 3 prize), a bonus of $50 will be awarded in addition to the prize money.
EdgeNet Topcoder Challenge Series
As part of the Topcoder EdgeNet Challenge series, we are excited to hear and learn from Topcoder Community members about your experience with this challenge, future challenges, and the EdgeNet platform. The winners of this challenge will have an opportunity to share their experience or be showcased as a top developer in EdgeNet challenges.
Topcoder may reach out to the winners, where they can opt to participate in additional marketing and promotional activities related to the EdgeNet Challenge Series.