Mobile App Usability Testing Challenge - 2

Hello Topcoders! Welcome to the Mobile App Usability Testing Challenge - 2. This is a Bug Hunt style competition; you’re required to test and experience the application as real users would do and submit feedback.

If this is your first competition on Topcoder do not fear - we have all the instructions, you need to participate in this challenge specification and in the challenge forums. Click the register button on this page to sign up, and then you’ll be granted access to the challenge forums where you can find documentation and ask questions.

IMPORTANT - This challenge has more than one prize placement! Prizes will be awarded as follows:

1st place: $500 | 2nd place: $400 | 3rd place $300

Submitters that do not take 1st, 2nd, or 3rd place will be paid $5 for each non-duplicate and verified issue up to a maximum of the 3rd place prize. An additional $5 will be paid for issues that require access to an actual vehicle.
 

ABOUT THE APPLICATION

The App allows Vehicle drivers to be more connected with their vehicle than ever before. The App offers everything from vehicle management, remote access, to meaningful assistance and support all in one place, so that your enhanced driving experience is nice and comfortable. 

 

The app lets you manage your vehicle on-the-go, from scheduling service, to making payments, to getting technical help at the push of a button. With app you can:

• View your manuals and how-to videos

• Schedule payments with Financial Services

• Contact your dealer and schedule service

• Speak to a Product Concierge, your vehicle tech expert

• View your MBFS lease or finance info and enroll in Auto Pay

• Access Roadside Assistance

• Edit your account and contact information

• Manage multiple vehicles

• Discover exclusive perks at partner resorts

• See the latest news from the company

 

Some of the features require  access to an actual vehicle. Issues raised for such features will have an additional $5 and 5 points awarded.

Read the challenge specification carefully and watch the forums for any questions or feedback concerning this challenge. Let us know if you have any questions in the challenge forum!

 

WHAT TO TEST

The objective of this challenge is to validate application functionality, usability (UI/ UX) of all the workflows in the app. Particularly focusing on workflows around Product info, search, user account, and the ease with which you could navigate within the app to access required details, etc. 

 

• Test URL: Testing the Live app download & install from Apple App store and Google Play store 

The above links are to the US version of the app. In case you do not have access to the app in your  country. you can use the version of the app that is available in your country. Please update the app download link that you used in the Admin-Copilot Notes field of the issue in case you are using a different version of the app than the version given above.

• Primary target device: Both iOS and Android devices (incl. Mobiles and Tablets)

• Browsers: N/A

• UX Suggestions are accepted and in-scope

• Content bugs are in-scope excluding the issues observed in re-directed URLs, articles, blog posts, FAQ, Privacy, Terms and Conditions pages. 

• If you need to create a user profile to test/experience the workflows, do it but DON'T test the features/workflow requiring you to make payments, scheduling service, sending any data via contact forms, etc. They are out of scope and WILL NOT be reimbursed.  

• Important: Because you are testing a live App, DO NOT Test/send data via contact forms. They are out of scope. 

• An edge case would be anything that does not reflect typical user behavior. They are accepted according to the impact to the end-user and based on the workarounds available.

 

HOW TO CREATE NEW BUG REPORTS

1. Create an issue in the Jira for the bug/issue you find

2. Please ask for access to Jira through forums

3. Issues/Bugs found in this App must created here:
   https://topcoderqaas.atlassian.net/secure/RapidBoard.jspa?projectKey=MMM&rapidView=160
   DON'T use any other link to create new issues OR submit a document, they won't be counted and won't be paid.

4. Use  https://drive.google.com/open?id=1AsNtmLKOPBzUXb_7X4w3iKPLwXN6yHm6  to upload screenshots, videos related to your bug here. Create a folder under your user name and then create  individual folders for each issue. You must provide the link to the respective issue folder when creating the issue in JIRA. 

ISSUE REPORTING GUIDELINES

For each report of a limitation or bug or usability improvement suggestion, we need the following information:

 

• Steps to reproduce, including all relevant information to assist diagnosis and resolution (detailed description of the steps to reproduce the issue incl. the test data and inputs keyed-in) 

• Current results (observed behavior) before the bug is fixed 

• Expected results, after the bug is fixed 

• If it is a UI bug attach Screenshots (highlight the area where the bug is observed) | Functional Bug – recorded Videos (if any) | Crash - Console/Crash Logs 

• Attach high-level labels to the logged issue. In case you are selecting multiple labels (Platform/Device), please provide screenshots for each and every Device/Platform selected; otherwise the issue is considered INCOMPLETE and will be REJECTED. 

• Update details pertaining to platform/os version, device type (iPhone, iPad, Tablet, etc) and model, Frequency (number of times the issue observed), Location, and Priority.  

• If it is a comparison, you must provide the URL and Screenshot/video of that location. 

 

IMPORTANT NOTICE

• Follow the standard Topcoder Bug Hunt Rules

• If you do not properly document your bug reports, they will likely be rejected due to lack of information or documentation. If you submit the same bug in multiple areas/pages, (for instance, Same validation issue of a form can be found in different pages/sections) you will likely get credit for the original bug report only. The others will all be closed as duplicates.

• If you duplicate an issue on a platform or browser that hasn’t been tested yet, you should create a new issue and add a link/reference in the issue description to the existing issue number. Our copilot will review these items and consolidate them later. Please don’t make adjustments or change labels of existing issues logged by other competitors.

• DON'T RE-OPEN the issues in the review phase and anyone who RE-OPENS a ticket will be disqualified from the challenge.

• DON'T EDIT OR ATTACH FILES to the issues in the review phase and anyone who is doing this will be disqualified from the challenge.

• If Mobile and Tablet testing are available DON'T create the same issue on different platforms; instead, merge them into one; All the others will be marked as Duplicate.

• You must not edit the bug report once created, so make sure you enter all the details at the time you create the issue, otherwise, your issue will be moved to the end of the queue. If you really need to edit an issue you must use the comments section for this (i.e. add a comment to describe any changes you want to make to the issue), and we'll decide whether the changes are major enough to move the issue to the end of the queue. You are allowed to add screenshots in the comments section though, assuming your issue report contains all the details when created.

• You must specify the test data you have used in the 'Reproduction Steps', All the issues will be marked as 'Incomplete', if the correct test data is not provided.

• There will be no appeals phase. The decision of PM/co-pilot for validity and severity of each filed issue will be final.

 

SCORING

• Scoring will be based on the number of bugs by weight.  Be sure to correctly attach a weight to your bug. The delivery team has the right to change a severity at their discretion.

• Scoring is directly proportional to weight:  Weight 1 = 1 point, Weight 10 = 10 points.

• Only verified issues will be counted.  Tickets created for enhancements or that are not bugs will not be counted. Duplicate issues will be closed and not counted. Log issues according to the guidelines above issues that do not follow these guidelines may be rejected due to lack of information.

• For challenge scoring, the user with the most verified issues will be selected as the winner. Submitters that do not take 1st, 2nd, or 3rd place will be paid $5 for each non-duplicate and verified issue up to a maximum of the 3rd place prize. If two users submit the same issue, the user that submitted the issue first will receive credit.

• Please focus on functionality and Usability testing based on the requirements, all bug reports based on your own assumptions will be rejected.

• We'll decide whether a bug is valid or not based on our knowledge about the app, it's totally up to us.

 

ISSUE WEIGHTS

• System failure/ defects (Functional Issues) - 10 points

• UX Enhancements/ Design Improvement Suggestions - 5 points

• UI issues and Content Error - 2 points

• An additional 5 points will be added to any issue that requires access to an actual vehicle.

TIPS

Some of the tips helpful for the contest:

• Submitting what is obviously the same issue multiple times with small variations will only annoy the reviewer that has to sort through all the issues and will only count as one issue anyway. If it's less obvious if it is the same issue or not, use your best judgment and the reviewers will do the same.

• Keep an eye on the issues being submitted by other participants to minimize the time you may be spending on duplicate efforts. Knowing what has already been reported will allow you to better focus your time on finding yet undiscovered issues.

• Double check your steps to reproduce and test cases to make sure they are clear. Make sure your steps include the creation of any necessary data.

Final Submission Guidelines

Submit all your bugs directly to JIRA. When you are done with your submissions please submit a .txt file to Topcoder Online Review using the Submit to this Challenge button BEFORE the Submission phase ends. In this file include:

 

• JIRA Username

• JIRA mail id

• topcoder handle (The one displayed in the top right corner near the profile picture)

 

• ALL THE SUBMISSIONS WITHOUT ABOVE INFORMATION WILL BE REJECTED AND WON’T BE PAID.

• IMPORTANT: Submit the above details before the Submission Phase ends. If you can't submit due to technical difficulties within the Submission Phase, please email your submission with above details to support@topcoder.com 

• Participants who haven't submitted will not be paid

• DON'T use any other link to create new issues OR submit a document, they won't count and won't be paid.