Challenge Overview
A previous challenge has implemented a set of REST APIs for handling video assets, including storing them and managing them (create, retrieve, update, delete). We also built a sample RSS scraper that parses data out of configured feeds and puts video assets in to the data store using the video REST API. This challenge will implement a new parser to parse out FoxNews.com feeds
Existing Code
The existing application is in Gitlab and access will be provided through links in the forum.
Scraper
The scraper will be implemented as a configurable delayed job. The job will run at a configurable interval and will read in RSS feeds, looking for assets added since the last time it ran. Each asset will be parsed and placed into the data store using the REST API.
The scraper will be configured with:
* A URL to the RSS feed
* A category to use when adding videos
* A provider value to use when adding videos
Sample data
For this challenge, please target the data in the Fox News feed here:
http://feeds.foxnews.com/foxnews/video
The category value should be "News" for the scraper, and the provider will be "Fox News"
* The image in the description should be used as the thumbnail for the video.
* The video URL should be the URL to the *video* on the playback page, not just the playback page itself. Here's an example:
http://foxnews-f.akamaihd.net/i/BrightCove/694940094001/2016/06/01/694940094001_49233,05642001_4923297220001,06993001_4923297220001,05632001_4923297220001,05693001_4923297220001,10861001_4923297220001,.mp4.csmil/master.m3u8
The m3u8 link is part of the iFrame on the video page
* The duration should be parsed from the m3u8 manifest.
Heroku deploy
Your deployment documentation should extend the existing documentation for the Node services and should cover how to deploy the newly created job to Heroku to run at a regular interval on a separate dyno from the service.
Code refactoring
The Fox News parser can share a lot of code with the existing Wall Street Journal parser. Don't just copy / paste that code - please make sure to properly refactor the code so that the parsers share any common functionality, like parsing the title and description. The only differences should be parsing out the video URL and duration from the video playback page.
Existing bugs
There may be a few minor bugs in the code right now - these are not your responsibility to fix, unless they block implementation of the requirements above. It would be appreciated if you logged them as part of your submission.
Submission format
Your submission should be a Git patch file against commit hash f9090ce94db2c9f8fd7f987ccb940a5529989045. Make sure to test your patch file before submitting!
Deployment document
Your patch file should update the README with information about configuring and using the Fox News parser.
Existing Code
The existing application is in Gitlab and access will be provided through links in the forum.
Scraper
The scraper will be implemented as a configurable delayed job. The job will run at a configurable interval and will read in RSS feeds, looking for assets added since the last time it ran. Each asset will be parsed and placed into the data store using the REST API.
The scraper will be configured with:
* A URL to the RSS feed
* A category to use when adding videos
* A provider value to use when adding videos
Sample data
For this challenge, please target the data in the Fox News feed here:
http://feeds.foxnews.com/foxnews/video
The category value should be "News" for the scraper, and the provider will be "Fox News"
* The image in the description should be used as the thumbnail for the video.
* The video URL should be the URL to the *video* on the playback page, not just the playback page itself. Here's an example:
http://foxnews-f.akamaihd.net/i/BrightCove/694940094001/2016/06/01/694940094001_49233,05642001_4923297220001,06993001_4923297220001,05632001_4923297220001,05693001_4923297220001,10861001_4923297220001,.mp4.csmil/master.m3u8
The m3u8 link is part of the iFrame on the video page
* The duration should be parsed from the m3u8 manifest.
Heroku deploy
Your deployment documentation should extend the existing documentation for the Node services and should cover how to deploy the newly created job to Heroku to run at a regular interval on a separate dyno from the service.
Code refactoring
The Fox News parser can share a lot of code with the existing Wall Street Journal parser. Don't just copy / paste that code - please make sure to properly refactor the code so that the parsers share any common functionality, like parsing the title and description. The only differences should be parsing out the video URL and duration from the video playback page.
Existing bugs
There may be a few minor bugs in the code right now - these are not your responsibility to fix, unless they block implementation of the requirements above. It would be appreciated if you logged them as part of your submission.
Submission format
Your submission should be a Git patch file against commit hash f9090ce94db2c9f8fd7f987ccb940a5529989045. Make sure to test your patch file before submitting!
Deployment document
Your patch file should update the README with information about configuring and using the Fox News parser.