Release Assembly - CSFV User Photo Upload and Retrieval Improvement

Project Overview

This project, the CSFV Community Gaming website project will provide the web site infrastructure and integration of the games. This will be a web site that is available on the Internet, and also will be delivered in a form where the server and the games can be installed and used behind a firewall.

Challenge Task Overview

We are having few issues when dealing with user photos in the website :

• The current logic does not work properly when we enable client-side caching of images. We are always using same URL for user image, so when caching is enabled then when user upload new image, he will able to see it across the website until cache is cleared or image caching is expired.
• We are using gm node module to resize, crop and render user photo in the website. We execute the gm each time we request the image (when it is not cached), with the increase of user base in the website it started to affect performance, consuming lots of CPU and memory. We ran performance testing and Large Amazon EC2 VM will complain about high cpu usage and it sometimes cause the website application to hang.

We are fixing this in this challenge.

Challenge Requirements

Current Logic

Existing user photo logic works as following :

• We have two storage options to store graphics and videos in CSFV application :
  • Amazon S3 : used in production and staging
  • Local storage : used in Offline (behind firewall) installation of CSFV website.
• When user uploads a photo :
  • It firsts call POST /social/attachments/:id to upload the file and save it as attachment, the route is handled in app.js and go through multiple middlewares (depends on storage option).
  • After user photo is stored as attachment, it calls POST /userPrivateProfile/saveConfiguration to associate the uploaded photo with user entity.
• When browser retrieves a photo it will call GET /photo/:username/?:width/?:height and execute getProfilePicture function.
  • Current code hardcode a 30 seconds as caching for image
  • It download the image, resize/crop it if dimensions are provided, and return it to user. Disregard if it is in S3 or local storage.

Overview of New Logic

We think of new logic as follow :

• When uploading an attachment, we will generate all supported photo sizes.
• When browser retrievs a photo using GET /photo/:username/?:width/?:height we redirect the user to GET /social/attachments/:id/?:width/?:height
• In GET /social/attachments/:id/?:width/?:height will pull the image, set caching headers, and send image to user.
• This solution will always gurantee that browser will render latest uploaded photo with proper caching header. Also we eleminate ability for user to run gm when they retrieve photos.

Implementation notes of the New Logic

You will make the following changes in this challenge :

• For upload logic :
  • Add new endpoint to upload photo "POST /photo/attachments/:id"
  • Update app.js to include new app.post() call for "POST /photo/attachments/:id"
    • It should also have a new middleware called 'setGeneratePhotoFlag', this function should set a flag 'generatePhotos' to true, this will be passed to "uploadMiddleware" middleware.
  • Update both LocalStorageUploadMiddleware and S3UploadMiddleware to have the following : 
    • It should have a new configuration 'photoSizes', each item should be in this format : "width:height", i.e. ["100:200", "240:230", "50:40"]
      • Parse the field and validate it during initialization of the controller, it is a required field.
    • If 'geneatePhotos' flag is present and equal to true, then generate images of all configured 'photoSizes'
      • The generated photos should be stored using same file path/name and appending the height/width to the end of file name. i.e. uploaded image name is /image.png and photo size is 40:120 then file path will be /image.png-40-120
      • Resize and crop image logic exists is located here.
• Update getProfilePicture as follow (This function serve the endpoint GET /photo/:username/?:width/?:heigh)
  • Change setExpiration function to read the expiration from configuration.
  • Change this conditional statement to check be if(!user || (user.photo != null && user.photo != undefined)).
    • This block will return im
  • Remove this logic from the function. The new logic should redirect user to GET /social/attachments/:id/?:width/?:height
    • :id - represents the user.photo
    • :width - (optional) represents the width path parameter passed to getProfilePicture
    • :height - (optional) represents the height path parameter passed to getProfilePicture
• Update getAttachment function as follow, use same logic we removed from getProfilePicture
  • Retrieve attachment from socialNetworkService using the provided :id
  • Append width/height to filePath (if provided) and try to retrieve the file, If it does not exist, then return error to use "File does not exist!"
  • If it exists, read the image, set cache control header same as done in setExpiration in getProfilePicture. Expiration seconds value should also be read from configuration.
  • return the image to caller.
  • We dont' do any cropping or resizing of images in this method.
• Update csfv_frontend_module/views/userpublicprofile/PublicProfileCommonData.jade and csfv_frontend_module/views/userprivateprofile/PrivateProfileCommonData.jade to use the new "POST /photo/attachments/:id" endpoint to attach user photo.

New Admin Script

We need an admin script to run to fix existing photos so it works with the new functionality implemented in this challenge :

• Add new file to csfv_frontend_module called : fixUserPhotos.js
• Add function 'fixPhotos' to the file to handle fixing the existing user photos.
• The function should do the following :
  • retrieve users from DB
  • iterate over each user one by one
  • retrieve attachment of user.photo
  • validate if attachment exists, if not set user.photo = null and save it
  • if attachment exists, retrieve supported sizes and validate that there is an image for each size, if not exist, generate image for that file, and store it.
  • The logic to be used in this function already described in the new logic above.
• Print 'success' to console, otherwise, return error message describing the problem occurred. and Exit the code.

General Notes

• Use best TopCoder development practices.
• Ask in forums if you have any doubts, copilot will answer your questions, even the technical ones.
• Cleanup the code.
• Document your code. and Clearly document the new configuration fields.

Testing

• Use user profile to upload photo. When hovering over user photo it will show you 'edit' button, you can upload photo from there.
• We are using the /photo/:username/:height/:width everywhere in the website, you can scan the views for this route (search for /photo/), navigate to that view to test the rendering.
• Make sure to test against both Amazon S3 and Local Storage.

Final Submission Guidelines

Submission Deliverables

Below is an overview of the deliverables:

• Patch file of the changes, include the hash commit used as basis for the work.
• A complete and detailed deployment document explaining how to deploy the application including configuration information.

Final Submission

For each member, the final submission should be uploaded to the Online Review Tool.