Challenge Overview
Project Overview
The client (GBG) is looking for the community to develop a monitoring system that collects data (e.g. performance metrics/statistics data) from laptop/desktop computers, and provides a cloud-hosted administrative console for admin users to query/visualize data collected in real time. Based on the statistics/metrics data, administrators may make decisions regarding new laptops, additional memory, software patches and upgrades, etc.
The system will consist of two sub-systems - the Time Series Database to receive and persist metric data, the Agent that runs on client laptop/desktop computers for collecting local data and sending to Time Series Database.
Notes, the original architecture suggesting a separate module - Admin Console, which is reduced, we are expecting a simpler solution.
Competition Task Overview
Detailed Requirements
Windows Platform consideration only: The following statement may talk broadly for different platform, but for this challenge, you just need to architecture for the windows platform, but keep the potential support of OS X in mind. Generally, I think the logic for retrieving system specific metric and reporting to OpenTSDB should be same.
Time series metric data are collected from individual computers, and Agents sub-system is responsible for this task.
An Agent will be deployed on a computer (mainly desktop/laptop personal computer, but may also be more powerful server) and run in background to periodically collect system diagnostic/statistical metric data (CPU/memory utilization, etc.). The collected data will be sent over to OpenTSDB sub-system (via HTTP API) for persistence. Basically an Agent can be seen as the bridge between the computer's local operating system and the OpenTSDB.
In general, Agents are platform-specific, and each Agent implementation is supposed to work on its target hardware architecture and/or operating system, like Windows 7/8, OS X etc. However in order to minimize development and maintenance effort, it is proposed to use cross-platform programming languages/frameworks to implement cross-platform agent as much as possible.
A cross-platform agent will be implemented to target both Windows 7/8 and OS X.
The common agent implementation will be implemented using Python, and it will target on both Windows 7/8 and OS X platforms.
The proposed Windows/OS X Agent implementation is shown on "Windows and OS X Agent Interface Diagram", and the details are available in SDS 5.2.1.
See the SDS Section 4 “Data Model” for the description of the models.
See the SDS Section 5 “Process Architecture” for the description of the services and how the services are used for addressing the requirements.
Note that the implementation interfaces are for reference only, for this challenge, you are encouraged to improve them as appropriate, to make it more professional to address the requirements.
Besides the general architecture specification, this is the important requirements to address clearly:
- Please specify how to define the Schema for OpenTSDB
- Please specify clearly how to retrieve the metric data.
- Please specify clearly how to invoke the HTTP API by using python.
- Please specify clearly how to wrap the solution for installing into Windows 7/8 easily.
Technology Overview
- OpenTSDB 2.0.1
- Apache HBase 0.98.9
- OpenTSDB HTTP API v2
- Python 3
- psutil
- pyspectator
- py2exe
- py2app
- Operating Systems
- Windows 7/8
Final Submission Guidelines
Submission Deliverables
- Application Design Specification
- Class Diagrams
- Sequence Diagrams
- Assembly Specifications
Submission Guidelines
- For each member, the final submission should be uploaded via the challenge detail page on topcoder.com.
- Submissions must include all deliverables as specified in 3 Submission Deliverables.
- The final submission will be reviewed using the standard Online Review Member Architecture Scorecard.