Challenge Overview
Challenge Overview
Welcome to the Performance Automation Framework - Simple Flow Challenge
Your task in this challenge is to implement two Java CLI tools - control script and simple flow execution script.
Project Information
We are building several tools in this project to test application performance by loading the target application in various browsers and reporting various metrics (lighthouse and web vitals). We will be developing three java tools:
-
Control script
-
Simple flow utility
-
Scripted flow utility
Control script will be a CLI tool that takes an xls file as input and calls the simple or scripted flow tools for each row in the spreadsheet.
Simple flow tool will take another excel as input with list of sites that need to be loaded and will report the load performance data
Scripted flow tool will also take an xls input file and load target sites (same as simple flow), but will also have options to do various operations on the site (using selenium), like entering data in forms, click on elements, wait for operations to complete etc, and finally report the performance data.
Control script will be executed either locally, or from a Jenkins job (defining the Jenkins job template is in scope of the project).
Technology stack
-
Java
-
Selenium
Assets
Controller script excel and simple flow excel files are posted in the forums.
Task Details
You have two tasks in this challenge:
-
Build the control tool
-
Build the sample flow tool
It is up to you whether to implement them as actual two CLI tools or just have the control script call sample flow as a class method - either approach is fine.
Control script will have the following inputs:
-
Path to the input xls file with columns: FlowName, FlowType (Simple, Scripted), mode (values: Desktop, Tablet, Mobile), Switch (ON, OFF), Browser (chrome, firefox, ie, edge), Cache (Incognito, Cache), Time (seconds), Debug (Yes, No)
-
Or values for a single execution (ie values for a single row from xls - flow name, flow type, mode, browser, cache, time, debug parameters)
Workflow of the controller script is:
-
For every flow which switch is ‘ON’, it should execute each script in a separate browser parallelly.
-
Based on the field ‘Mode’, script should execute in desktop, tablet, mobile browser emulator.
-
Based on the field ‘Browser’, script should execute in the respective browser.
-
Each script should run for a specified period using the value in field ‘Time’.
-
For every iteration, it should use either same browser with cache or open new browser with no cache using value in field ‘Cache’,
-
For ‘debug’ mode yes, it should print results in local Excel file, For ‘debug’ mode no, it should send message to a Kafka topic
Simple flow utility will take an xls file as a parameter and it will have two sheets:
-
Scenario - contains a list of sites to load (columns: Steps, Description, Transaction, Selection Type, Data, URLStringCheck, Lighthouse)
-
Data - contains the data for each of the scenarios
Selection type field can have the following values:
-
Single: Should pick the first row of data from the data spreadsheet
-
Random: Randomly pick data from the list
-
Sequential: For every iteration pick data sequentially
Data field specifies which column to read the data for the scenario from the Data spreadsheet.
Simple flow utility should for every step check and capture below data:
-
Http response
-
Check if “URLStringCheck” value is present in the loaded url
-
Status (Pass or Fail): Fail only if destination url doesn’t contain URLStringCheck, or http response is 4xx or 5xx
-
URL
-
Time
-
Transaction name
-
Lighthouse report data
-
First Contentful Paint
-
Time to Interactive
-
Largest Contentful Paint
-
Total Blocking Time
-
Cumulative layout shift
-
-
Web vitals data:
-
Time to First Byte
-
Full Load Time
-
Useful links for lighthouse and web vitals:
Submission
Submit the full source code, readme documentation file with details on how to run the tool, sample outputs.