FTA - Bug Hunt (QA)

This challenge will have 24hr for the registration (during this phase no one will see details) and 4 days for bug hunting.

Challenge Objectives

Welcome to Bug Hunt challenge.

• There is no coding or designing needed.

• All you need in order to participate is to open in your browser two versions of the application and check that new version works the same as an old version.

Note: The challenge prizes are mentioned for top 3 placement prizes. For other submitters, there are additional prizes to be awarded per valid bugs.

About the Application

We have an existent Flash application provided by the client and we have rewritten this application with React. And we want to make sure that new React application works the same as the old Flash application.

We will provide you with links to live Flash and React applications which work with mock API which returns the same data for them. So we can run these two applications side by side and check that new one behaves the same.

Assets

• Links to live Flash and React applications.

• Brief verification guide with the main workflows.

• GitLab link to report issues.

• Server request verification guide with the instruction on how to check that requests to the server match.

• The source code of the Flash and React apps.

What to Test (Scope)

Browsers in scope

• Google Chrome 69+ on Windows 10

• Internet Explorer 11 on Windows 10

Live app comparison

• Open old Flash app and React app in any browser in scope and try to find a place where new React app behaves not like old Flash application. Note, the UI in general of the React app is close to the Flash app but not exactly repeats it. Design mismatches are NOT COUNTED as issues unless something is broken in React app or leads to poor user experience in comparison with the Flash app. There are also some UI issues in the Flash app which we didn’t reproduce in React, these differences are also not counted as an issue.

• Follow the Brief verification guide provided on the forum for a starting point. It describes the main workflows in the application and also gives examples of test data to enter to get from the mock API some special results like no search results or force server-side error.

• When you run the application you would have to set some Application parameters. For testing use ONLY combinations of these parameters listed in the Brief verification guide. Any issues found with other combinations of Application parameters wouldn’t be counted.

• There are some sections in Brief verification guide marked with a red icon . These functionalities have the most complex logic so you could point attention to them, as they are most likely to have some bugs/issues.

Server requests comparison

This is an additional way of finding issues which couldn’t be seen just by visually going through the apps. You can use this way to find issues others cannot see.

• As Flash app uses AMF format over RPC, you cannot easily see requests in the browser console. You can follow Server request verification guide to know how to check and compare requests of Flash in React.

• There are some differences in the way server requests are sent by Flash and React. Please, follow Server request verification guide to understand which mismatches are valid issues and which are not.

• Responses from mock API shouldn't be tested, only requests sent by the app.

How to Create a New Bug Report

1. Create an account on GitLab (if you do not already have one)

2. You can get access to the GitLab repo using the link, provided in the challenge forum. If you are still getting 404 error, raise a question on the forum and copilot will try to help you.

3. Issues/Bugs found in this application/App must create here: https://gitlab.com/fta-flex-to-html/fta-react/issues. DON'T use any other link to create new issues OR submit a document, they won't be counted and won't be paid.

4. Please label issues with the appropriate bug type. The labels can be viewed here: https://gitlab.com/fta-flex-to-html/fta-react/labels.

Issue Reporting Guidelines

For each report of a limitation or bug, we need the following information:

1. Always add to the title of the issue the section in the application where it happens. See existent issues for example https://gitlab.com/fta-flex-to-html/fta-react/issues. All possible sections names are marked with [] inside Brief verification guide, like [View Forms], [Document Processing] and so on.

2. Always mention which Application parameters set you used for testing: “Field User (1)”, “Admin User (2)”, “Deep Linking (3)”.

3. Mention the browser and version you used for testing if you think the issues is specific to the browser.

4. Steps to reproduce, including any needed information (Must list all the steps that need to reproduce the bug, DON'T list only the URL without test data).

5. What is the wrong behavior in React app

6. What is the correct behavior in the Flash app

7. If it is a UI bug attach a Screenshots (Mark the area where the bug is). Attach the screenshot of the correct behavior in Flash app.

8. If it is a Functional Bug attach videos for both: how React app works wrong, and how Flash app works correctly. You can attach videos directly on GitLab, if not use services like www.screencast.com or https://monosnap.com/. Don’t use www.youtube.com to host the videos. For crashes, attache Console/Crash Logs for React app.

IMPORTANT NOTE:

Missing or Incorrect details to ANY of the above fields will mark the bug report as INCOMPLETE. 

Issue Weights and Scoring

• Scoring will be based on the number of bugs by weight.  Be sure to correctly attach a label to your bug to identify the type and related points. The delivery team has the right to change a severity at their discretion.

• Only verified issues will be counted. Tickets created for enhancements or that are not bugs will not be counted. Duplicate issues will be closed and not counted. Log issues according to the guidelines above issues that do not follow these guidelines may reject due to lack of information.

• For challenge scoring, the use with the most verified issues/points will be selected as the winner. If two users submit the same issue, the user that submitted the issue first will receive credit.

• Please focus on functionality/UI testing based on the requirements, all bug reports based on your own assumptions will be rejected.

• The following points will be awarded as per issue category:

• Functional Issues         - 10 Points

• Server Request Issues    - 10 Points

• User Interface Issues         - 5 Points

• Performance Issues         - 5 Points

• Usability/UX Issue         - 2 Point (missing tooltips are usability UX issues)

• Content Bug             - 1 Point

• The 1st place submission must raise at least bugs worth 100 points.

• The 2nd place submission must raise at least bugs worth 75 points.

• The 3d place submission must raise at least bugs worth 50 points.

• Submitters that do not take 1st, 2nd, and 3rd place will be Paid $1 Per Earned Point for each non-duplicate and verified issue up to a maximum of the 3rd place prize. If two users submit the same issue, the user that submitted the issue first will receive credit.

Important Notice

• Follow the standard Topcoder Bug Hunt Rules.

• If you do not properly document your bug reports, they will likely be rejected due to lack of information or documentation. If you submit the same bug in multiple areas/pages, (for instance, Same validation issue of a form can be found in different pages/sections) you will likely get credit for the original bug report only. The others will all be closed as duplicates.

• DON'T RE-OPEN the issues in the review phase and anyone who RE-OPENS a ticket will be disqualified from the challenge.

• DON'T EDIT OR ATTACH FILES to the issues once it has been submitted and anyone who is doing this will be disqualified from the challenge.

• You must not edit the bug report once created, so make sure you enter all the details at the time you create the issue, otherwise, your issue will be moved to the end of the queue. If you really need to edit an issue you must use the comments section for this (i.e. add a comment to describe any changes you want to make to the issue), and we'll decide whether the changes are major enough to move the issue to the end of the queue.

• You must specify the test data you have used in the 'Reproduction Steps', All the issues will be marked as 'Incomplete', if the correct test data is not provided.

• Keep an eye on the issues being submitted by other participants to minimize the time you may be spending on duplicate efforts. Knowing what has already been reported will allow you to better focus your time on finding yet undiscovered issues.

• There will be no appeals phase. The decision of PM/Copilot for validity and severity of each filled issue will be final.

Final Submission Guidelines

Submit all your bugs directly to GitLab. When you are done with your submissions please submit a .txt file using the “Submit” button before the submission phase ends. In this file include:

• Topcoder handle (The one displayed in the top right corner near the Profile picture)

• GitLab handle used to raise the issues.

- ALL THE SUBMISSIONS WITHOUT ABOVE INFORMATION WILL BE REJECTED AND WON’T BE PAID.

- IMPORTANT: Submit the above details before the Submission Phase ends. If you can't submit due to technical difficulties within the Submission Phase please email your submission with above details to support@topcoder.com.