How to test iOS & android apps on AWS Device farm: Part I

Most of the app development teams rely on manual testing for their applications. This process can be extremely time consuming & complex because there are a large variety of devices on the iOS market & Android market. Testing becomes even more difficult because of multiple variations of models, OS versions, carrier customizations and more.

AWS Device Farm addresses this challenge by letting developers test their application on a  shared fleet of 2500+ devices or on your own private device lab in the cloud. It does not use simulators, but real non-rooted devices.

Login or register an AWS account

Login to your AWS account, if you don’t have one then register yourself for a new AWS account. You will need to set up permission for the IAM user to access Device Farm by creating a new access policy in IAM.

The accounts we are going to provide in Topcoder QA challenges usually have these policies pre-configured so developers can directly jump in &  start creating a project to run a test.

Getting Started with device farm

After login, the first step is to select the device farm option from the dashboard page. The device farm service can also be searched from the Find Services if needed.

Selecting the device farm takes us to the device farm dashboard page from where we can create a new project for testing our App. We can also select an existing project for testing.

A topcoder QA challenge usually has a project created in the device farm.  The participants are required to select the project mentioned in challenge specifications & start testing after setting some configurations that we are going to learn in this article.

Creating a new project

Click the “+ Create a new project” to create a new project. Give your project a name & press the Create project button.

Types of test

There are two types of testing that we can perform using Device Farm:

Automated Testing: Allows us to test our app in parallel against a massive collection of physical devices in the AWS Cloud.

Remote Access:  Gesture, swipe and interact with devices in real time, directly from a web browser.

Automated tests

Let’s deep dive into an automated test & learn how we can perform this.

Creating a new run

From the newly created project dashboard select the Automated tests tab & press the Create a new run button. This opens a Create a new run window.

Choose Application

Select the button with android & apple icons, then upload your files to be tested, usually we have to upload APK or IPA files & give a run name to your project run.

Configure

Once the test APK is uploaded, configure the test & select the test type you would like to perform. Here we are going to test the app using the Built-in: Fuzz test. In the Advanced Configuration make sure that the “Enable Video Recording” & “Enable App Performance Data Capture” are checked.

Select devices

Form the select devices tab select the Device pool to select the list of devices on which test has to be performed.

To create a custom list of devices on which test has to be performed, select the Create a new device pool option. Refer to the Topcoder QA challenge specification to get the list of devices on which test has to be performed & then based on that create a new device pool.

Specify device state

Specify settings to simulate real-world scenarios. The location of the device can also be simulated by setting the Device location value with lat. & lng. coordinates of a location.

Review and start run

Review the test setting & change the Execution timeout value if required. If everything looks good then click the Confirm and start run button to start running the test.

The test run we just created is listed on the Automated tests tab. Click the test name “my-test-run” to check the progress of our test.

Once the test run is completed the screenshots, parsing result and other details get updated. In a Topcoder QA challenge developers may have to upload these data along with the other required files. Refer to the challenge deliverables sections for a list of things to submit.

After the running test on all the listed devices, the test status is displayed. The  Repeat run button gets activated now that allows repeating the test.

This is how we can perform an automated test on all devices quickly & effortlessly. We don’t even require to own all those devices & configure different mobile OS versions.

In the next blog article, we will see how to perform Remote access test that allows us to swipe, gesture, and interact with a device through a web browser in real time in order to test functionality and reproduce customer issues.