Challenge Summary
Welcome to "HP Data Quality ETL Application Design Challenge". In this challenge, we need your help to create new look and feel for our Extract Transform and Load (ETL) application that will allow user to moves data between a source file or database and a destination directory or database.
At the end of this challenge, We are looking for the [topcoder] design Community to help us planning the look and feel of our application.
01. Dashboard Screen
03. Create/Edit Project Screen
04. Project Details Screen
06. Create/Edit Job Screen
- Note: Please number your screen to simplify client review.
00. Login Screen
01. Dashboard Screen
02. Settings Screen
03. Create/Edit Project Screen
04. Project Details Screen
05. Job Details Screen
06. Create/Edit Job Screen
- Note: Please number your screen to simplify client review.
Background Overview
The HP Data Quality team is tasked with moving large amounts of data on a regular basis. They would like to development an Extract Transform and Load tool that allows them to move file and relational data on a regular basis and create a monitoring service which can display the status of the currently scheduled and recently completed jobs. Often they are moving data between regular and HDFS file systems.
The application will need to handle the following data source types:
- Fixed Width Text Files
- Delimited Text Files (such as csv files)
- JSON Files
- XML Files
- Excel Files
- Relational Databases (Oracle, SQL Server, MySQL)
- Columnar Databases (Redshift and Vertica)
- NoSQL Databases (MongoDB)
- Hadoop Databases (Hive)
Screen Requirements:
We need to create the following screens for the application:
00. Login Screen
WF Reference: Login
- This tool will have both ETL capabilities and job monitoring capabilities.
- Users will log into the tool to see all the jobs that are currently scheduled as well as those in the future.
- Provide screen interactions when user enter wrong credential.
01. Dashboard Screen
WF Reference: Dashboard-user (User section)
- Default screen after user successfully login to the apps.
- From this screen, user can open a project to find the job details and create jobs.
- Show summary of all activity in the apps, such as:
-- Total Projects
-- Total Jobs
-- Running Execute Job
-- Upcoming Job
-- Completed Job
- Show list of Active and Completed Projects.
- In the project list section, user can view them as list or grid.
- User can drag and drop any project details in grid and list view
- Arrange the project by their status: with warning, running, modified, others
- User will be able to see summary of projects:
-- Project Name
-- Project Description
-- Created
-- Modified
-- Created By
-- Modified By
-- Status: Running/Total Jobs
-- and Action buttons (Open project details, Edit Project, Delete Project)
02. Settings Screen
WF Reference: Settings (User section)
- User can configure their component version from this screen.
- There are two tabs: input files and output files.
- Then expandable sections to avoid to get lost having too much visible information.
- "+ Add File", means to add library files to the referred version
- "Edit - Save/Cancel", means to edit version label and be able to modify or delete file paths manually.
- "Delete", to delete a version
- "+ Add Version", to include a new version relation
- "+ Add Type", to include a new file type that needs to be supported
- User can add new file or database type that needs to be supported in this screen.
03. Create/Edit Project Screen
WF Reference: CreateProject and EditProject (User section)
- User can create a new project from this page.
- Simple form with the following field needs to be added in this screen:
-- Project Name
-- Project Description
-- Creation Date
-- Created By
- Once user successfully create a project, they will be redirected to project details screen and can start manage all jobs there.
04. Project Details Screen
WF Reference: ProjectDetails
- This screen should show project description summary and show summary of "Project Status" and "Jobs"
- Show all "Running Jobs", "Completed Jobs" and "Upcoming Jobs" in list and grid view
- Please note that all Job detail list here can be dragged and drop.
- The Job details should have the following content:
-- Job name
-- Description
-- Created
-- Modified
-- Created by
-- Modified By
-- Status
-- Schedule
-- Action Buttons (Schedule Job, Edit Job, Delete Job)
05. Job Details Screen
WF Reference: JobDetails
- Show all information related to the Jobs here.
- Contains the job list/grid with job details.
- From here you can open a project to find the job details and create jobs.
- Please create a version where the Job details don't have a schedule set (WF Reference "JobDetails-noSchedule") and a version where there is an error found in the job details (WF Reference "JobDetails-Error")
- Information related to jobs detail can be found in the collapsible section (user can expand/collapse any informations) :
-- Details
-- Progress
-- Schedule
-- History
- User can edit Schedule for the jobs by clicking edit button in Schedule section (WF Reference "JobScheduler-Edit"):
-- User can define start and end date of a job
-- At the bottom area, user can define time for each day
-- We need to provide users with the ability to schedule jobs to operate on a recurring basis.
-- This screen should include fields to allow users to pick a start date, and end date, days of the week and time of day.
06. Create/Edit Job Screen
WF Reference: CreateJob-Description, CreateJob-Workspace, CreateJob-MappingInterface, CreateJob-JobRunner, EditJob-Description, EditJob-Workspace, EditJob-MappingInterface, EditJob-JobRunner
Job Description
- User can enter description of the job in this area
- Fields needs to be included in this section are:
-- Job Name
-- Job Description
-- Creation Date
-- Created By
-- Modification Date
-- Modified By
- Once Job Description done, click Next
Workspace
- Once the basic details of a job have been declared user should be able to enter a workspace where they can enter the details of their data migration job. There should be several parts to the workspace screen:
-- Component List - The ETL tool will also need a list of components. In the toolbox there should be input and output versions of each of the components listed above. (e.g Input Delimited Text File, Output Delimited Text File, Input XML File, Output XML File, etc..).
-- Workspace - A job is created by dragging components into the workspace. Each job will have two components - a source and a destination.
- We need to create set of screen interactions on how to use the workspace area:
-- Please provide screen interactions regarding drag and drop features (drag componentlist item to object fields)
-- Please provide screen interactions once the item successfully dropped to the object fields (notice the field and value area, create it for source and destination)
-- Please provide screen interactions where there are more than 2 toolbar item in each source and destination area.
-- Please provide screen interactions when user remove the component list item from the workspace.
- Once the workspace is done, click Next.
Mapping Interface
- One of the most important elements of the job is the mapping interface where source and destination fields are mapped to each other.
- When an input and output component both have a schema definition, users should be able to bring up a mapping screen which will allow users to map input fields to output fields and do some basic transformations. It would be great if this mapping could done visually. There should be a line in the User Interface connecting an input field to an output field.
- On the mapping interface we'll also support some basic transformations. This will allow users to modify data as it changes between input and output. Here are the functions that will be supported initially (click the f(x) link):
-- Trim() - removes all whitespace
-- Right(integer numOfChars) - removes a certain number of characters from the right side of the input string
-- Left(integer numOfChars) - removes a certain number of characters from the left side of the input string
-- FindAndReplace(String stringToFind, String stringToReplace) - replaces characters in the source data with the replacement string.
-- DateToString(String dateFormat)
-- NumberToString (String numberFormat)
-- CurrencyToString (String currencyFormat)
-- Index(String stringToFind) - returns an integer index value of the designated string or -1 if the value isn't found.
-- These functions will available at the field level for the relevant input source data type. For example, Trim(), Right(), Left() are only really relevant to Strings. They wouldn’t apply to numeric input. DateToString would only apply to fields of type Date, etc.
- User can add as many as they want new field in this area.
Job Runner
- As jobs are created it will be important to actually execute a job in real time to verify results.
- The app should provide some progress indications as a data is read from the source and written to the destination.
Important:
- Keep things consistent. This means all graphic styles should work together.
- All of the graphics should have a similar feel and general aesthetic appearance
Documentation
- Wireframes
Branding Requirements
- You may use HP related colors and styles but please do not include HP logos in the Storyboards or application. Usage of the official HP branding invites scrutiny from corporate marketing.
- http://www.hp.com/
Screen Size
Desktop: 1280 px width with height adjusting accordingly
Target Audience
- HP Data Quality personnel - They are fairly technical but may not be hardcore programmers. Used to using tools to move data around.
Judging Criteria
- How well you plan the user experience and capture your ideas visually.
- How well you implement the challenge requirements.
- Completeness and accuracy of the designs.
- Cleanliness of your graphics and design.
- Creativity and ease-of-use is key to the success as it must be engaging to users.
Stock Artwork Policies
You can only use stockphoto in this challenge, stock icons not allowed for this challenge.
- Stock Artwork policies: https://help.topcoder.com/hc/en-us/articles/217481408-Policy-for-Stock-Artwork-in-Design-Submissions
- Latest Design Policies: http://help.topcoder.com/hc/en-us/categories/202610437-DESIGN
Submission & Source Files
Preview Image
Please create your preview image as one (1) 1024x1024px JPG or PNG file in RGB color mode at 72dpi and place a screenshot of your submission within it.
Submission File
Submit JPG/PNG for your submission files.
Source Files
Please provide all original source files of the submitted design. Files should be created in Adobe Photoshop (layered PSD file) or Adobe Illustrator (AI File).
Final Fixes
As part of the final fixes phase you may be asked to modify your graphics (sizes or colors) or modify overall colors. We may ask you to update your design or graphics based on checkpoint feedback.
At the end of this challenge, We are looking for the [topcoder] design Community to help us planning the look and feel of our application.
Round 1
Initial Design for client review01. Dashboard Screen
03. Create/Edit Project Screen
04. Project Details Screen
06. Create/Edit Job Screen
- Note: Please number your screen to simplify client review.
Round 2
All requirements like stated in challenge details with client feedback applied00. Login Screen
01. Dashboard Screen
02. Settings Screen
03. Create/Edit Project Screen
04. Project Details Screen
05. Job Details Screen
06. Create/Edit Job Screen
- Note: Please number your screen to simplify client review.
Background Overview
The HP Data Quality team is tasked with moving large amounts of data on a regular basis. They would like to development an Extract Transform and Load tool that allows them to move file and relational data on a regular basis and create a monitoring service which can display the status of the currently scheduled and recently completed jobs. Often they are moving data between regular and HDFS file systems.
The application will need to handle the following data source types:
- Fixed Width Text Files
- Delimited Text Files (such as csv files)
- JSON Files
- XML Files
- Excel Files
- Relational Databases (Oracle, SQL Server, MySQL)
- Columnar Databases (Redshift and Vertica)
- NoSQL Databases (MongoDB)
- Hadoop Databases (Hive)
Screen Requirements:
We need to create the following screens for the application:
00. Login Screen
WF Reference: Login
- This tool will have both ETL capabilities and job monitoring capabilities.
- Users will log into the tool to see all the jobs that are currently scheduled as well as those in the future.
- Provide screen interactions when user enter wrong credential.
01. Dashboard Screen
WF Reference: Dashboard-user (User section)
- Default screen after user successfully login to the apps.
- From this screen, user can open a project to find the job details and create jobs.
- Show summary of all activity in the apps, such as:
-- Total Projects
-- Total Jobs
-- Running Execute Job
-- Upcoming Job
-- Completed Job
- Show list of Active and Completed Projects.
- In the project list section, user can view them as list or grid.
- User can drag and drop any project details in grid and list view
- Arrange the project by their status: with warning, running, modified, others
- User will be able to see summary of projects:
-- Project Name
-- Project Description
-- Created
-- Modified
-- Created By
-- Modified By
-- Status: Running/Total Jobs
-- and Action buttons (Open project details, Edit Project, Delete Project)
02. Settings Screen
WF Reference: Settings (User section)
- User can configure their component version from this screen.
- There are two tabs: input files and output files.
- Then expandable sections to avoid to get lost having too much visible information.
- "+ Add File", means to add library files to the referred version
- "Edit - Save/Cancel", means to edit version label and be able to modify or delete file paths manually.
- "Delete", to delete a version
- "+ Add Version", to include a new version relation
- "+ Add Type", to include a new file type that needs to be supported
- User can add new file or database type that needs to be supported in this screen.
03. Create/Edit Project Screen
WF Reference: CreateProject and EditProject (User section)
- User can create a new project from this page.
- Simple form with the following field needs to be added in this screen:
-- Project Name
-- Project Description
-- Creation Date
-- Created By
- Once user successfully create a project, they will be redirected to project details screen and can start manage all jobs there.
04. Project Details Screen
WF Reference: ProjectDetails
- This screen should show project description summary and show summary of "Project Status" and "Jobs"
- Show all "Running Jobs", "Completed Jobs" and "Upcoming Jobs" in list and grid view
- Please note that all Job detail list here can be dragged and drop.
- The Job details should have the following content:
-- Job name
-- Description
-- Created
-- Modified
-- Created by
-- Modified By
-- Status
-- Schedule
-- Action Buttons (Schedule Job, Edit Job, Delete Job)
05. Job Details Screen
WF Reference: JobDetails
- Show all information related to the Jobs here.
- Contains the job list/grid with job details.
- From here you can open a project to find the job details and create jobs.
- Please create a version where the Job details don't have a schedule set (WF Reference "JobDetails-noSchedule") and a version where there is an error found in the job details (WF Reference "JobDetails-Error")
- Information related to jobs detail can be found in the collapsible section (user can expand/collapse any informations) :
-- Details
-- Progress
-- Schedule
-- History
- User can edit Schedule for the jobs by clicking edit button in Schedule section (WF Reference "JobScheduler-Edit"):
-- User can define start and end date of a job
-- At the bottom area, user can define time for each day
-- We need to provide users with the ability to schedule jobs to operate on a recurring basis.
-- This screen should include fields to allow users to pick a start date, and end date, days of the week and time of day.
06. Create/Edit Job Screen
WF Reference: CreateJob-Description, CreateJob-Workspace, CreateJob-MappingInterface, CreateJob-JobRunner, EditJob-Description, EditJob-Workspace, EditJob-MappingInterface, EditJob-JobRunner
Job Description
- User can enter description of the job in this area
- Fields needs to be included in this section are:
-- Job Name
-- Job Description
-- Creation Date
-- Created By
-- Modification Date
-- Modified By
- Once Job Description done, click Next
Workspace
- Once the basic details of a job have been declared user should be able to enter a workspace where they can enter the details of their data migration job. There should be several parts to the workspace screen:
-- Component List - The ETL tool will also need a list of components. In the toolbox there should be input and output versions of each of the components listed above. (e.g Input Delimited Text File, Output Delimited Text File, Input XML File, Output XML File, etc..).
-- Workspace - A job is created by dragging components into the workspace. Each job will have two components - a source and a destination.
- We need to create set of screen interactions on how to use the workspace area:
-- Please provide screen interactions regarding drag and drop features (drag componentlist item to object fields)
-- Please provide screen interactions once the item successfully dropped to the object fields (notice the field and value area, create it for source and destination)
-- Please provide screen interactions where there are more than 2 toolbar item in each source and destination area.
-- Please provide screen interactions when user remove the component list item from the workspace.
- Once the workspace is done, click Next.
Mapping Interface
- One of the most important elements of the job is the mapping interface where source and destination fields are mapped to each other.
- When an input and output component both have a schema definition, users should be able to bring up a mapping screen which will allow users to map input fields to output fields and do some basic transformations. It would be great if this mapping could done visually. There should be a line in the User Interface connecting an input field to an output field.
- On the mapping interface we'll also support some basic transformations. This will allow users to modify data as it changes between input and output. Here are the functions that will be supported initially (click the f(x) link):
-- Trim() - removes all whitespace
-- Right(integer numOfChars) - removes a certain number of characters from the right side of the input string
-- Left(integer numOfChars) - removes a certain number of characters from the left side of the input string
-- FindAndReplace(String stringToFind, String stringToReplace) - replaces characters in the source data with the replacement string.
-- DateToString(String dateFormat)
-- NumberToString (String numberFormat)
-- CurrencyToString (String currencyFormat)
-- Index(String stringToFind) - returns an integer index value of the designated string or -1 if the value isn't found.
-- These functions will available at the field level for the relevant input source data type. For example, Trim(), Right(), Left() are only really relevant to Strings. They wouldn’t apply to numeric input. DateToString would only apply to fields of type Date, etc.
- User can add as many as they want new field in this area.
Job Runner
- As jobs are created it will be important to actually execute a job in real time to verify results.
- The app should provide some progress indications as a data is read from the source and written to the destination.
Important:
- Keep things consistent. This means all graphic styles should work together.
- All of the graphics should have a similar feel and general aesthetic appearance
Documentation
- Wireframes
Branding Requirements
- You may use HP related colors and styles but please do not include HP logos in the Storyboards or application. Usage of the official HP branding invites scrutiny from corporate marketing.
- http://www.hp.com/
Screen Size
Desktop: 1280 px width with height adjusting accordingly
Target Audience
- HP Data Quality personnel - They are fairly technical but may not be hardcore programmers. Used to using tools to move data around.
Judging Criteria
- How well you plan the user experience and capture your ideas visually.
- How well you implement the challenge requirements.
- Completeness and accuracy of the designs.
- Cleanliness of your graphics and design.
- Creativity and ease-of-use is key to the success as it must be engaging to users.
Stock Artwork Policies
You can only use stockphoto in this challenge, stock icons not allowed for this challenge.
- Stock Artwork policies: https://help.topcoder.com/hc/en-us/articles/217481408-Policy-for-Stock-Artwork-in-Design-Submissions
- Latest Design Policies: http://help.topcoder.com/hc/en-us/categories/202610437-DESIGN
Submission & Source Files
Preview Image
Please create your preview image as one (1) 1024x1024px JPG or PNG file in RGB color mode at 72dpi and place a screenshot of your submission within it.
Submission File
Submit JPG/PNG for your submission files.
Source Files
Please provide all original source files of the submitted design. Files should be created in Adobe Photoshop (layered PSD file) or Adobe Illustrator (AI File).
Final Fixes
As part of the final fixes phase you may be asked to modify your graphics (sizes or colors) or modify overall colors. We may ask you to update your design or graphics based on checkpoint feedback.
Please read the challenge specification carefully and watch the forums for any questions or feedback concerning this challenge. It is important that you monitor any updates provided by the client or Studio Admins in the forums. Please post any questions you might have for the client in the forums.