Browser Based UI Automation Testing Ideation Challenge

Please note that the client will be choosing the winners, so the review phase will be longer than usual.

 

Challenge Objective

Welcome to the Browser Based UI Automation Ideation Challenge! The goal of this challenge is to determine the best tooling and technologies to build automated tests that run in Internet Explorer 11 as the primary target browser, and other popular browsers.

• Generate ideas on the best set of tools to run automated tests, given requirements below.

• Focus on IE11 support, but will need to be cross-browser capable.

• Focus on functional testing, with other levels of testing considered.

 

Round 1

You should submit a solution to describe your idea with higher-level details to qualify for the checkpoint prizes. 

Round 2

Please submit your full solution. 

Technical Requirements

The following are requirements for the automation strategy:

1. IE11 is the most important browser.

2. Chrome, and Safari are additional target browsers. 

3. Tests would ideally be written once and work for all supported browsers with minimal browser-specific code.

4. Must support functional testing at a minimum. 

5. Preference will be given to solutions that make recommendations for unit testing, E2E testing, performance testing, and load testing.

6. Browser to device connection

   1. REST API and WebSocket

   2. CGI scripts

7. Stability is critical (Reduce flaky tests.)

8. Soft assertions

9. Parallel execution preferred.

10. Screenshot/video capture preferred.

11. Support for Test Design Patterns (Page Objects Pattern, Screenplay Pattern, etc.).

 

Framework Requirements

We are looking for tools that satisfy the following requirements:

1. Low amount of Framework maintenance

2. Preference is given to industry-standard tools and tools that have robust community support

3. Licensing - we are open to recommendations. For licenses with a cost, we’d prefer a per-seat model as opposed to a named model.

 

Important Notes

• Our User Interface is hosted by a small embedded device. While the UI can be run from anywhere, the data served to it will always come from one of these small local devices (typically on our local network which is firewalled-off from the outside world). An online sample of our UI can be provided.

• The app is web-based (Chrome, IE11, Safari) and built upon React, Material-io, and Redux via Google Material.

• We are not opposed to Cloud-based tooling if it is the absolute best, but something onsite might better support our needs.  

Format

Submit a document that has these major sections. There are no minimum or maximum length requirements, but please be comprehensive in your analysis. 

• Summary: Briefly describe the overall approach to solving the problem. For instance, did you find a single tool, or will we need to use multiple? 

• Tool analysis: describe the tools you are recommending and why. Please note which automation requirements and tool requirements they satisfy and the ones they dot. 

• Architecture: If we need to integrate multiple tools, how do you recommend we do this? 

• Overall Analysis: How will this support cross-browser testing needs, and the concerns provided in the “Important Notes” Section?

 

Winner Selection

• There will be no appeal phase, and the winner selection is purely up to the client. 

 

What to Submit

We are looking for a written description which will layout the proposed solution that meets the requirements of the project (no code will be developed). 

• Markdown and PDF submissions are preferred, but we will accept .TXT and Word/Google docs as well.

• You are also encouraged to accompany your ideas with code snippets, links or anything else that may help us understand your idea(s).

• It should be written in English