Test for Success Challenge Series - QA Challenge 6.2

Welcome to Test for Success Challenge Series - QA Challenge 6.2. This is a weekly challenge of Test for Success Challenge Series to perform regression testing for one of the product application (MS StaffHub).

Note: The challenge prizes indicated is just the place holder which is total challenge prize amount. The prize money is per test cases basis. The prize money with each test case is tagged into gitlab issue tracker. The prize money will be paid for successful acceptance after review of test results submitted by tester. The test case wise prize money ranges from $20 - $150 prize.

Bonus: If you are a new submitter that sends an accepted submission to either one of this week's Test for Success challenges, you will have the chance to win an extra bonus of $50. One winner from each challenge will be randomly selected to win!

Challenge Objectives

• Regression testing using provided test cases - (Manual Test Case Execution).
• Test the iOS App, Android App, Windows Desktop App and Web App using specific Devices and OS.
• Fill out the test case execution report with PASS/FAIL/Blocked to execute test case under Test Result field and write down your comments under Obervations/Remaks/Assumptions field of the shared reporting Google Forms.
• Submit the Test Case execution report (using Google Form links provided in the reference document attached in the forum) by producing video with execution steps.
• For each test case you are executing, you have to attach the screenshot of your Device info screen. Important info that needs to show up in the snapshot has been clarified in the reference document attached in the forum.

Project Background

This is a Structured Test Case Execution Project for one of the product application (MS StaffHub) and there will be having series of challenges for every week to execute provided Test Cases. Every week there will be regression testing execution challenge with around 136 Regression Test Cases which has to be executed on iOS, Android, Windows Desktop and Web Browsers within 24 hours. The payment will be of Pay-Per-TestCaseExecution Model and the prizes range from $20 - $150 for each Test Case Execution.

Technology Stack

• .NET Technology
• iOS App
• Android App
• Windows Desktop App
• Web App - (Edge, IE11, Chrome, Safari Browsers)

Test Application Details

The information regarding the application access, setup and installation are kept inside the forum. Please register yourself to access the forum to get the application information for testing.

• iOS App
• Android App
• Windows Desktop App
• Web App URL

Challenge Requirements

Manual Test Case Execution should be performed by picking the test cases from the GitLab issues repository. The GitLab repository link can be found inside the forum. There are total 136 test cases (known as tickets) created under issues on GitLab repo. These details are available in forum, so please register the challenge to access the forum.

Fast Submission:

Please note that the Test Cycle is limited to 24 hours window for test results submission. From the forum, go to the provided Gitlab Link's issue list of available/open tickets for pickup and choose one of it and read it clearly and do the test execution and capture the test results video and reproducible steps for the bugs at any step and upload your video and test result at designated location. Reviewer will review your submission near real time and provide the feedback for your test results and you need to address the feedback within next 4 hours time. It is recommended to submit your test execution results quickly so that you can pick up new open ticket for test execution and compete to get more prize money. The prize money is per ticket basis; the more your tests and submit the results, the more you can earn the prize money.

Devices in Scope:

Following are the list of devices and browsers in scope of this test execution cycle.
- Please use and stick to any of the devices listed below. If you have any questions, please don't hesitate to ask in forum.
- The OS version mentioned is the minimum version required on the particular device.
- Phones are most preferred over Tablets - (so each tester is allowed to execute maximum of 3 test cases from any Android/iOS Tablets, if you are using one of the in-scope Android/iOS Tablets, but no limitation on the number of test cases you can execute on Phones).
- Android Emulator 4.4.4 is allowed for the in-scope devices listed below - (so each tester is allowed to execute maximum of 3 test cases from Android Emulator).
- AWS Device Farm is allowed and recommended for the in-scope devices listed below - (if you don't have in-scope devices with you, so we have recommendation for you to use real mobile devices in the AWS Cloud for manual testing).

• iOS:
  • iPhone 7 (10.3.2)
  • iPhone 6s+ (10.3.2)
  • iPhone 5s (10.3.2)
  • iPhone 6 (10.2.1)
  • iPad Air (10.3.1)
  • iPad Air 2 (10.3.2)
  • iPad Pro 9.3 (10.3.2)
  • iPad Pro 12.9 (10.3.1)
  • iPad Mini 4 (10.2)
• Android:
  • Samsung Galaxy S3 Neo (4.4.4)
  • Samsung Galaxy S7 Edge (7.0.1)
  • LG Flex 2 (5.1.1)
  • Samsung Galaxy Tab (6.0.1)
  • Sony Xperia (6)
  • Nexus 6P (7.1.1)
  • Moto G, 4th Gen (7.0.1)
  • One Plus 3T (7.0.1)
  • Huawei Honor 6x (6.0.1)
  • Nexus (7.1.1)
  • Pixel XL (7.1.2)
• Windows: Desktop App
• Web Browsers in Scope: Latest stable browsers - Edge, IE11, Chrome, Safari

How to Participate:

• Register into the challenge and open the challenge forum to get the instructions to download the builds and setup the devices for test execution, test cases, test case execution report Google Form and bug report Google Form.
• Navigate to the GitLab repository provided on the forum to register your GitLab ID to get access to GitLab repo. The following steps need to be followed for the testing.
• Step 1 - Choose the ticket labeled as "tcx_OpenForPickup".
• Step 2 - System will auto change the status as "tcx_Assigned".
• Step 3 - Work on the chosen ticket and submit your test results - video (video should record only the steps of the test cases, you pause when not testing and play to record when testing) and steps to reproduce the bugs if any:
  • Prepare the Test Case Google Form report with the screenshot of your Device info screen and also provide a recording for test case execution. If the test case execution takes longer time, you can split the video. Video Name should have the step numbers covered (if you are splitting).
  • If you identify any issues log them in the repository. It is important to note which test case and on which step the issue occurred. If someone already logged that issue, you can mention that ticket number, don't log duplicates.
  • Upon completion of your test cases and logging of issues attach the test case execution report to the issue you assigned yourself.
• Step 4 - Change the ticket status to "tcx_ReadyForReview".
• Step 5 - Reviewer will review your test results submission by adding "tcx_Review" label, and if accepted the status should be changed to "tcx_FixAccepted" and if not the status should be changed to "tcx_Feedback", and "tcx_Review" label should be removed when the status changed into "tcx_Feedback" status.
• Step 6 - If the ticket status is "tcx_Feedback", tester has to fix the review feedback before continue to work on the new chosen ticket, and then repeat the step 3 to 4.

Note: Continue above steps until all test cases have been executed or GitLab issues have been resolved.

Additional Rules and Payment Details:

• A participant is not supposed to hold the test case for more than 2 hours of time. Else, it will be unassigned from the participant. Similarly the review feedback needs to be resolved within 4 hours, otherwise the ticket submission will be rejected and the ticket will be open for pickup by others. (Please prepare your submission very carefully before submit it, the ideal case here is that your submission would be qualify for just one round of review, then you don't need to worry about this 4hrs rule)
• You will be awarded the listed prize amount for successfully executing the test cases for that test case and accepted by reviewer. If you successfully complete multiple sets of test cases, you will be awarded a prize for each one completed.
• If you execute a test case and miss an issue that is found by the reviewer, then you are on risk losing payment for that test case. It requires you to be thorough in testing and show clear verification steps in the video.
• For incomplete submissions or tickets/bugs having review feedback will receive penalty on ticket price. This will reduce 10%, 25%, 50% of ticket price for each failed submission and 50% is the most deduced price. So it is recommended to read the reference document carefully and submit the report with all necessary information without fail.
• The challenge submission that was scored more number of accepted tickets on GitLab by reviewers wins and in case of tie on accepted number of tickets, then the challenge submission that was uploaded earlier wins.

Winning Strategy:

• Register the challenge and participate early to understand the App flow and the Test Cases.
• You need to assign yourself the unassigned/open tickets only, please don't work on tickets assigned to others.
• Prepare the videos and screenshots for the current test case execution.
• Submit the Test Case Google Form report and Bug report Google Form (if found any bug) on required template/format for review.
• Once the current assigned ticket is ready for review, then you can work on next open ticket by assigning yourself.
• For a ticket, if you receive any feedback from reviewer, then work on it and submit quickly to pass review.
• For a ticket, if it is accepted by reviewer, then you will get paid the prize amount associated with the test case title.
• You can work as many tickets you can complete within 24 hours timeline to win more prizes.

Final Submission Guidelines

Submit all your bugs directly to GitLab. When you are done with your submissions please submit a .txt file using the "Submit" button before the submission phase ends. In this file include following details:

• TopCoder Handle: <>
• Gitlab ID: <>
• List of accepted issues (GitLab links) and Bug ID separated by comma: <[Gitlab Issue Link],[Bug ID]>

You could refer below instructions how to get your info:

• TopCoder handle (The one displayed in the top right corner near the Profile picture)
• GitLab handle (Login to GitLab and click on the Profile picture > Your Profile. Check the URL https://gitlab.com/[Your Username] )

Important Notes:

• DON'T use any other link to create new issues or submit as document, they won't count and won't be paid.
• DON'T use youtube or public shared location for uploading your videos and screenshots.