Sechi Integration Worker Apps Assembly

Project Overview

Create 3 (almost same) time-based background worker applications on Heroku platform to act as connector bridges to 3 vendor services (Xero, Deputy and Vend).  This Assembly will demonstrate how records created in Vendors can be automatically published to message broker.

Competition Task Overview

We had run one PoC challenge earlier, you can get concept from that but be care of many difference in this spec.

Your task is to create 3 separate heroku apps for the 3 different vendor services (Xero, Deputy and Vend), the 3 apps are almost same, see following required features for all:

1. Architecture

1.1 Use Heroku and Node.js as main platform, refer PoC winning submission code.

1.2 No web UI but only one background worker to act all.

1.3 OAuth are all long term access

1.4 Config by env variables like PoC code done.

1.5 All data transferred(Poll, Publish) should be JSON.

1.6 Logging all request url/header(except common and Authorization, only log what you set, such as "If-Modified-Since" for Xero API)/body, response code/body, error and exception into console(logplex)

1.7 Use Heroku Toolbel commandline to start/stop worker and view logs. document to deployment guide.

2. Worker

The worker will poll data periodically from vendor service to get new records and publish them to broker queue.

2.1 Functions in worker can be scheduled by "setInterval".

2.2 Poll data from API

2.2.1 Only Poll new data and don't publish repeat to queue. So each request should carry one timestamp (different format and position in APIs, see 2.2.3). ignore timestamp in first request to get all existing records.

2.2.2 Integrate which object from which Vendor, should be generic and config in one JSON file(not env variables). such as
xero:[receipts],
xero.payroll:[employees,timesheets]
These just mockup here, detail format is open to you.

2.2.3 Required objects
See attachment APIs.PDF

2.2.4 Keep state(timestamp, last object id if required?...) to postgresql to avoid broken when dyno restart. should check and read when worker dyno start.

2.3 Publish data to broker queue

2.3.1 Use heroku CloudAMQP add-on as broker. There's only one single broker for the 3 apps.

2.3.2 One kind of object one Queue, Queue is named by "vendor:object type" format such as "xero:receipts", "xero.payroll:employees". Don't delete/recreate queue like PoC.

2.3.3 One object one message, don't publish collections which got from poll directly.

2.3.4 All datetime format should be converted to standard ISO-8601 format(2014-07-23T08:39:14Z) when publish to queue.

2.3.5 Don't publish repeat record(except updated records) to queue, see 2.2.1 Note.

2.3.6 Web UI to view data stored in the broker queue: Use CloudAMQP dashboard(Note it's very slow, need wait after clicking) to handle the queue on broker, Need document details in deployment guide like PoC done. If you use CloudAMQP directly, need describe clearly.

3. Setup Test Data

Xero: https://www.xero.com/ (Create Australian organisation)
Deputy: http://www.deputy.com/
Vend: http://www.vendhq.com/

4. Deployment guide

Should be detailed about setup/config/validate the requirements.
Each app includes own deployment guide

Documentation Provided

Final Submission Guidelines

Submission Deliverables

A complete list of deliverables can be viewed in the TopCoder Assembly competition Tutorial at: http://apps.topcoder.com/wiki/display/tc/Assembly+Competition+Tutorials 

Below is an overview of the deliverables:

• 3 separate apps per requirement.
• Each includes a complete and detailed deployment documented explaining how to deploy the application including configuration information.

Final Submission

For each member, the final submission should be uploaded to the Online Review Tool.

Development Environment

Heroku
Nodejs