Test for Success Challenge Series - QA Challenge 8 (Bug Hunt)

Welcome to Test for Success Challenge Series - QA Challenge 8 (Bug Hunt). This is a weekly challenge of Test for Success Challenge Series to perform exploratory testing for one of the product application (MS Teams - Shift(int) and StaffHub).

Note: The challenge prizes are mentioned for top 5 placement prizes. For other submitters, there are additional prize money to be awarded per valid bugs or issues or suggestions created in the system and approved by client after review - please refer issue weight and scoring section.

Challenge Objectives

• Exploratory testing where client is looking for the feedback about the product how it can be made better and fins any flaws in the product.
• Test the iOS App, Android App, Windows Desktop App and Web App using specific Devices and OS.
• Create issues in GitLab with proper repro steps.
• Submit the screen recording video of bug repro steps and also provide useful bug screenshots if applicable.
• You have to attach the screenshot of your Device info screen. Important info that needs to show up in the snapshot has been clarified in the reference document attached in the forum.

Project Background

This is an un-structured and exploratory testing for one of the product application (MS Teams - Shift(int) and StaffHub) and there will be having series of challenges for every week. The product application has to be tested on iOS, Android, Windows Desktop and Web Browsers within 24 hours. The payment will be of Pay-Per-Bug Model and the prizes will be awarded if you will log valid and eccepted issues by client.

Technology Stack

• .NET Technology
• iOS App
• Android App
• Windows Desktop App
• Web App - (Edge, IE11, Chrome, Safari Browsers)

Note: Safari on Windows is not in Scope

Test Application Details

The information regarding the application access, setup and installation are kept inside the forum. Please register yourself to access the forum to get the application information for testing.

• iOS App
• Android App
• Windows Desktop App
• Web App URL

Challenge Requirements

There will no test cases from client and tester will have to test the product application as end user perspective and provide the feedback about the issues and any features if not working. If you find an issue, then it should be logged as defects/suggestions/improvements/security issues. The GitLab repository link can be found inside the forum. You need to create issues on GitLab repo, so please register the challenge to access the forum.

Devices in Scope:

Following are the list of devices and browsers in scope of this cycle.
- Please use and stick to any of the devices listed below. If you have any questions, please don't hesitate to ask in forum.
- The OS version mentioned is the minimum version required on the particular device.
- Phones are most preferred over Tablets - (so each tester is allowed to log maximum of 3 bugs from any Android/iOS Tablets, if you are using one of the in-scope Android/iOS Tablets, but no limitation on the number of bugs you can log on Phones).
- Android Emulator 4.4.4 is allowed for the in-scope devices listed below - (so each tester is allowed to execute maximum of 3 bugs from Android Emulator).
- AWS Device Farm is allowed and recommended for the in-scope devices listed below - (if you don't have in-scope devices with you, so we have recommendation for you to use real mobile devices in the AWS Cloud for manual testing).

• iOS:
  • iPhone 7 (10.3.2)
  • iPhone 6s+ (10.3.2)
  • iPhone 5s (10.3.2)
  • iPhone 6 (10.2.1)
  • iPad Air (10.3.1)
  • iPad Air 2 (10.3.2)
  • iPad Pro 9.3 (10.3.2)
  • iPad Pro 12.9 (10.3.1)
  • iPad Mini 4 (10.2)
• Android:
  • Samsung Galaxy S3 Neo (4.4.4)
  • Samsung Galaxy S7 Edge (7.0.1)
  • LG Flex 2 (5.1.1)
  • Samsung Galaxy Tab (6.0.1)
  • Sony Xperia (6)
  • Nexus 6P (7.1.1)
  • Moto G, 4th Gen (7.0.1)
  • One Plus 3T (7.0.1)
  • Huawei Honor 6x (6.0.1)
  • Nexus (7.1.1)
  • Pixel XL (7.1.2)
• Windows: Desktop App
• Web Browsers in Scope: Latest stable browsers - Safari(Mac Only), Edge, IE11, Chrome

How to Participate:

• Register into the challenge and open the challenge forum to get the instructions to download the builds and setup the devices for exploratory testing
• Navigate to the GitLab repository provided on the forum to register your GitLab ID to get access to GitLab repo.
• If you find an issue in the application, then follow steps below to submit the issue for client review.
  • If tester finds a bug, then keep a recorded video and document repro steps of the bug.
  • Create bug under GitLab issues with the required format of bug title and bug description. Please assign the bug to yourself.
  • Please check existing issues before logging your new issue to avoid any duplicate issue.
  • Provide proper issue labels (More details on Reference Document attached to the Forum)
  • Note that your labels might be modified by reviewers/clients/copilots.
  • If the issue is not accepted, then the reviewer will close the ticket and provide short description about why it was rejected and submitter cannot edit the existing one. Submitter could resubmit a new one which will be treated new ticket(check below Additional Rules section). 
  • If it is a duplicate issue or invalid issue, then it will be closed by the reviewer.
  • One exception is if the previous ticket is not accepted as per some step wrong or anything else, the latter duplicated tickets should be reviewed.

Additional Rules:

• Follow the standard Topcoder Bug Hunt Rules (if any conflict please ask in forum)
• If you do not properly document your bug reports, they will likely be rejected due to lack of information or documentation. If you submit the same bug in multiple areas/pages, (for instance, Same validation issue of a form can be found in different pages/sections) you will likely get credit for the original bug report only. The others will all be closed as duplicates.
• If you duplicate an issue on a different platform that hasn’t been tested yet, you should create a new issue and add a link/reference in the issue comments section about the Original issue number. Our copilot will review these items and consolidate them later. Please don’t make adjustments or change labels of existing issues logged by other competitors.
• DON'T RE-OPEN the issues closed by reviewer and anyone who RE-OPENS a ticket will be disqualified from the challenge.
• DON'T EDIT OR ATTACH FILES to the issues Once it has been submitted and anyone who is doing this will be disqualified from the challenge.
• You must not edit the bug report once created, so make sure you enter all the details at the time you create the issue, otherwise, your issue will be moved to the end of the queue. If you really need to edit an issue you must use the comments section for this (i.e. add a comment to describe any changes you want to make to the issue), and we'll decide whether the changes are major enough to move the issue to the end of the queue. 
• You must specify the test data you have used in the 'Reproduction Steps', All the issues will be marked as 'Incomplete', if the correct test data is not provided.
• There will be no appeals phase. The decision of PM/co-pilot for validity and severity of each filed issue will be final.

Issue Weight and Scoring:

• Scoring will be based on the number of bugs by weight.  Be sure to correctly attach a weight to your bug.
• The review team has the right to change a issue category, priority and severity at their discretion.
• Only verified issues by reviewer and accepted by client will be counted. Duplicate issues will be closed and not counted.
• Log issues according to the guidelines above. Issues that do not follow these guidelines may reject due to lack of information.
• For challenge scoring, the user with the most verified issues/points will be selected as the winner. If two users submit the same issue, the user that submitted the issue first will receive credit.
• Please focus on exploratory testing and log issues
• For any suggestions or improvements, please create separate issues.
• The following points will be awarded as per issue category:
  • Functional Issues: 10 Points
  • Security Issues: 10 Points
  • User Interface Issues: 5 Points
  • Usability/UX/Suggestion/Improvements: 2 Points
  • Content Issues: 1 Point
• The 1st and 2nd place submissions must raise at least bugs worth 150 points (75 points on Staffhub and 75 points on Shift-Int). Out of the 75 points, 50 points should be from mobile devices & 25 points from web/desktop
• The 3rd, 4th and 5th place submissions must raise at least bugs worth 110 points (55 points on Staffhub and 55 points on Shift-Int). Out of the 55 points, 30 points should be from mobile devices & 25 points from web/desktop
• For challenge scoring, the user with the most verified issues will be selected as the winner. Submitters that do not take 1st, 2nd, 3rd, 4th or 5th place will be Paid $2 Per Earned Point for each non-duplicate and verified issue up to a maximum of the 5th place prize. If two users submit the same issue, the user that submitted the issue first will receive credit.

Final Submission Guidelines

Submit all your bugs directly to GitLab. When you are done with your submissions please submit a .txt file using the "Submit" button before the submission phase ends. In this file include following details:

• TopCoder Handle: <>
• Gitlab ID: <>

You could refer below instructions how to get your info:

• TopCoder handle (The one displayed in the top right corner near the Profile picture)
• GitLab handle (Login to GitLab and click on the Profile picture > Your Profile. Check the URL https://gitlab.com/[Your Username] )

Important Notes:

• DON'T use any other link to create new issues or submit as document, they won't count and won't be paid.
• DON'T use youtube or public shared location for uploading your videos and screenshots.