Remote Expert Application - iOS and Backend Bug Hunt

Challenge Overview

Welcome to “Remote Expert Application - iOS and Backend BugHunt”. In this challenge, we want to build the search for issues in the Remote Expert iOS Application and the corresponding backend service.

This challenge will have 24hrs for the registration (during this phase no one will see details) and 24hrs for bug hunting. Details will be posted on Aug 28, 1:00 pm EDT.
 

CHALLENGE OBJECTIVES

• Exploratory or unstructured testing and reporting.
• Complete as many as you can in 24 hours.
• iOS, backend and integration

HOW TO PARTICIPATE

• Scope: Remote Expert application is in scope. iOS source code will be provided.
• App Builds: Available in the forums when we open the submission phase. Specific instructions for how to install are available in the reference document.
• Repository information will be provided in the forum after the initial 24h.

DEVICES IN SCOPE

• iPhone 6s and higher - portrait mode only
• Important detail about the iOS application.
  • ���The iOs application uses ARKit to make and receive video calls
  • To test this part of the code you will need 2 iOS devices with support to ARKit

Penalty on Duplicate Bug Reporting: If you log/report any duplicate bug, then you will receive penalty 2 Points per duplicate bug regardless of Bug type.
Penalty on Rejected Bug Reporting: If you log/report any bug without required information (insufficient info on bugs or missing repro steps or missing screenshot or missing video, etc.), then you will receive penalty 5 Points per rejected bug regardless of Bug type.

 

Additional Rules:

• Follow the standard Topcoder Bug Hunt Rules (if any conflict please ask in forum)
• If you do not properly document your bug reports, they will likely be rejected due to lack of information or documentation. If you submit the same bug in multiple areas/pages, (for instance, Same validation issue of a form can be found in different pages/sections) you will likely get credit for the original bug report only. The others will all be closed as duplicates.
• If you duplicate an issue on a different platform that hasn’t been tested yet, you should create a new issue and add a link/reference in the issue comments section about the Original issue number. Our copilot will review these items and consolidate them later. Please don’t make adjustments or change labels of existing issues logged by other competitors.
• DON'T RE-OPEN the issues closed by reviewer and anyone who RE-OPENS a ticket will be disqualified from the challenge.
• DON'T EDIT OR ATTACH FILES to the issues once it has been submitted and anyone who is doing this will be disqualified from the challenge.
• You must not edit the bug report once created, so make sure you enter all the details at the time you create the issue, otherwise, your issue will be moved to the end of the queue. If you really need to edit an issue you must use the comments section for this (i.e. add a comment to describe any changes you want to make to the issue), and we'll decide whether the changes are major enough to move the issue to the end of the queue. 
• You must specify the test data you have used in the 'Reproduction Steps', All the issues will be marked as 'Incomplete', if the correct test data is not provided.
• There will be no appeals phase. The decision of PM/Copilot for validity and severity of each filed issue will be final.

Issue Weight and Scoring:

• Scoring will be based on the number of bugs by weight. Be sure to correctly attach a weight to your bug.
• The review team has the right to change a issue category, priority and severity at their discretion.
• Only verified issues by reviewer and accepted by client will be counted - (valid no repro / on hold bugs are considered as accepted for final standings). All duplicate & rejected issues will be closed with penalty and not counted.
• Log issues according to the guidelines above. Issues that do not follow these guidelines may reject due to lack of information.
• For challenge scoring, the user with the most verified issues/points will be selected as the winner. If two users submit the same issue, the user that submitted the issue first will receive credit.
• For any suggestions or improvements, please create separate issues.
• The following points will be awarded as per issue category:
  • Video call Issues (integration and UI): 5 Points
  • Functional Issues: 5 Points
  • Other User Interface Issues: 2 Points
• The placement points will be calculated after receiving client feedback with accepted bugs only - (valid no repro / on hold bugs are considered as accepted for final standings). The client rejected bugs like Duplicate bugs will receive 5 Points penalty per bug where as by design, not a bug and any other bugs will receive no penalty.
• The 1st and 2nd place submissions must raise at least 8 client accepted bugs and at least total client accepted bugs worth 40 points.
• The 3rd and 4th place submissions must raise at least 8 client accepted bugs and at least total client accepted bugs worth 20 points.
• In case, no participants qualify for the placement prizes as per above rule, then one winner will be selected who have scored highest points after reviewers accepted bugs having at least 10 accepted bugs
• For challenge scoring, the user with the maximum points will be selected as the winner. Submitters that do not take 1st, 2nd, 3rd or 4th place will be Paid $1 Per Earned Point for each client accepted bugs up to a maximum of the 4th place prize. If two users submit the same points, then user that submitted the client accepted bug first will receive credit.
• Duplicate/Rejected Bugs Reporting: If you log/report any duplicate bug during review phase and after client review feedback, then you will receive penalty 2 Points per duplicate bug regardless of Bug type. If you log/report any bug without required information (insufficient info on bugs or missing repro steps or missing screenshot or missing video, etc.), then you will receive penalty 2 Points per rejected bug regardless of Bug type.

Final Submission Guidelines

1. Bug Report Process

Bug Report Format

For each report of a limitation or bug, we need the following information:

1. Steps to reproduce, including any needed information
2. Screenshots or Screen Capture (required)
3. Expected results after the bug are fixed
4. Current results, before the bug is fixed
5. Device and OS version

Important Notice:

If you do not properly document your bug reports, they will likely be rejected due to a lack of information or documentation. Also, make sure your bug reports are reasonably general.
If you submit the same bug that is seen in multiple screens, for instance, you will likely only get credit for the original bug report. The others will all be closed as duplicates.

2. Ticket Logging

You will log your tickets in the specified Gitlab repo. 

3. Tips

a) Submitting what is obviously the same issue multiple times with small variations will only annoy the reviewer that has to sort through all the issues and will only count as one issue anyway. If it's less obvious if it is the same issue or not, use your best judgment and the reviewers will do the same.
b) Put an eye on the issues being submitted by other members to minimize the time you may be spending on duplicate efforts. Knowing what has already been reported will allow you to better focus your time on finding yet undiscovered issues.
c) Double-check your steps to reproduce and test cases to make sure they are clear. Make sure your steps include the creation of any necessary data.

3. Submission Deliverables

You need to report your issues in Gitlab. Please submit a text file contains the bugs you reported to OR.

4. Final Submission

For each member, the final submission should be uploaded via the challenge detail page on topcoder.com.