Google Assistant | Bug Hunt

This challenge will have 24hr for the initial registration (during this phase no one will see details) and 2 days for bug hunting. Details will be posted on July 10th, 02:00 AM ET

 

Challenge Objectives

• Raise tickets in gitlab about issues that you find in our app

 

About the Application 

We have created an Action in Google Assistant (in simple terms, a bot for Google Assistant) that allows users to “register” themselves with our app using their google account and to receive notifications when their printer is running low on ink. There are two modules in scope:

• An Action On Google that allows users to “register” themselves with our app, meant for use in Google Assistant.

• Azure Services that carry out additional functions besides acting as the “backend” for our Action.

 

Assets

We will share the gitlab self sign up url with you in the contest forum on July 10th, 02:00 AM EST

 

What to Test (Scope)

• Deployment. Follow the instructions in the README files of each module and deploy them. You can report issues that you face with the deployment - important steps that should be mentioned but are not.

  • Note that for deployment, you need to have:

    • Google Account, particularly in DialogFlow

    • Azure account

    • A Google Cloud Service account

  • I would advise that until July 10th, 02:00 AM EST, you could try to create an Action On Google by following this codelab. That should help you familiarize yourself with DialogFlow.

• Conversation. At the end of the day, users will be interacting with our app by chatting with it so we want to get this right. Talk to our action as any user would and list out bugs that you come across. Note that we will not consider edge cases as bugs. Edge cases in this case would be defined as actions that users might rarely perform.

 

How to Create New Bug Report

1. Create an account on GitLab

2. We will share a self sign up link in the contest forum on July 10th, 02:00 AM EST. Sign up to our repository.

3. Once in, go through the source code and start testing. The link where you can report your issues will be shared in the contest forum on July 10th, 02:00 AM EST. It will the Issue tab of the same source code in Gitlab.

4. You don’t have to assign labels.

 

Issue Reporting Guidelines

 

For each report of a limitation or bug, we need the following information:

1. Steps to reproduce, including any needed information (Must list all the steps that need to reproduce the bug, DON'T list only the URL without test data)

2. Current result(s) before the bug is fixed

3. Expected result(s), after the bug is fixed

4. Provide as many details as you can. Screenshots will help immensely, else your issues might just be closed because it cannot be reproduced.

IMPORTANT NOTE:

• Missing or Incorrect details to ANY of the above fields will mark the bug report as INCOMPLETE. For example, Incorrect Steps, Missing Screenshot/Screencast (If it is a UI issue, you have to mark it on the screenshot), Incorrect Actual and Expected results etc.

 

Issue Weights and Scoring

• Scoring will be based on the number of bugs by weight. You DON’T have to assign weights. We will attach them during the review phase.

• Only verified issues will be counted.  Tickets created for enhancements or that are not bugs will not be counted. Duplicate issues will be closed and not counted. Log issues according to the guidelines above issues that do not follow these guidelines may reject due to lack of information.

• For challenge scoring, the user with the most verified issues/points will be selected as the winner. If two users submit the same issue, the user that submitted the issue first will receive credit.

• Please focus on functionality/UI testing based on the requirements, all bug reports based on your own assumptions will be rejected.

 

All bugs raised will be categorized as follows, with the corresponding weight

• Grammar Issues - 1 Point

• Conversation Flow issues - 5 Points

• Deployment Issues - 3 Points. Note that minor issues with deployment will attract 1 point only

 

At the end, users with the highest score and second highest score wins. Other users will get paid $5 per issue raised

 

Important Notice

• Follow the standard topcoder Bug Hunt Rules.

• If you do not properly document your bug reports, they will likely be rejected due to lack of information or documentation. If you submit the same bug in multiple areas, you will likely get credit for the original bug report only. The others will all be closed as duplicates.

• You can raise issues either with the Simulator or with your actual device. In case of the latter, don’t forget to take screenshots and upload them.

• DON'T RE-OPEN the issues in the review phase and anyone who RE-OPENS a ticket will be disqualified from the challenge.

• You must not edit the bug report once created, so make sure you enter all the details at the time you create the issue, otherwise, your issue will be moved to the end of the queue. If you really need to edit an issue you must use the comments section for this (i.e. add a comment to describe any changes you want to make to the issue), and we'll decide whether the changes are major enough to move the issue to the end of the queue. You are allowed to add screen shots in the comments section though, assuming your issue report contains all the details when created.

• You must specify the test data you have used in the 'Reproduction Steps', All the issues will be marked as 'Incomplete', if the correct test data is not provided.

• Keep an eye on the issues being submitted by other participants to minimize the time you may be spending on duplicate efforts. Knowing what has already been reported will allow you to better focus your time on finding yet undiscovered issues.

• There will be no appeals phase. The decision of PM/Copilot for validity and severity of each filled issue will be final.

Final Submission Guidelines

Submit all your bugs directly to GitLab. When you are done with your submissions please submit a .txt file using the “Submit” button before the submission phase ends. In this file include:

• Topcoder handle

• GitLab handle used to raise the issues.

 

Note that:

• ALL THE SUBMISSIONS WITHOUT ABOVE INFORMATION WILL BE REJECTED AND WON’T BE PAID.

• IMPORTANT: Submit the above details before the Submission Phase ends. If you can't submit due to technical difficulties within the Submission Phase please email your submission with above details to support@topcoder.com.

• Participants who haven't submitted will not be paid.

• DON'T use any other link to create new issues OR submit as document, they won't count and won't be paid.