Challenge Overview
For this challenge, we want to update the submission processor to add support for development submissions.
The submission processor consists in a chain of responsibility that is built by ProcessorFactory depending on the type of submission.
The Processor is responsible for updating legacy systems with information about the submission, so that, for instance, Online Review, Direct, and v2 api are aware of the submission. We ran an architecture challenge that describes the steps that are currently done by the v2 api to receive and process a development submissions. We need to perform all those steps as part of the development submissions feature of the processor. Please check Topcoder - Existing Development Submission Process.zip in forum for detail reference.
You need to update the ProcessorFactory to build an appropriate processor for develop submissions.
The ProcessorFactory creates a composite processor, in which each item is a processor that performs a discrete task. Some of these tasks can be reused for develop submissions, some may have to be modified.
Note, the current submission process for design submissions SHOULD NOT be impacted by these changes. If changes are made to the shared processors, please make sure to thoroughly test them with design submissions.
- PreProcessor
Is a processor that performs some setup work for the processing of the submission. It includes creating a transaction handler the objects that represent the upload and submission records that will later be saved to Informix. This processor may be used for development submissions, but the values pre-populated onto the informix upload and submission objects must be reviewed.
- UserFetchProcessor
Simply retrieves user information, can be used as is for development submissions.
- DesignZipDownloadProcessor
This processor downloads zip files from s3 to a local folder so that they can be processed. Design submissions require changes to the zip files, which is probably not needed for develop submissions (please confirm by looking at the v2 api documentation). The develop submissions need to be copied to nfs, so a variation of this processor will be necessary to download from s3 and put the file in the correct location in the nfs share.
- DesignZipSubmissionValidatorProcessor
This processor performs validations in the data and zip files. Since the develop challenge submissions validations are significantly different, probably the best approach would be to have a separate validator processor just for develop submissions, and leave this one just for design submissions. Please review the v2 api documentation to determine which validations are done today, and perform the same validations in the submission metadata. Also please check if the zip file is valid, and not empty.
- DeclarationGeneratorProcessor, FontsProcessor, StockArtProcessor, SubmissionDeclarationFileProcessor, PreviewValidationProcessor, DesignUnifiedZipProcessor, WatermarkProcessor, and S3UploadProcessor
Not applicable to develop submissions;
- PostgresMetadataProcessor
Saves metadata about the files uploaded to Postgres. A similar process needs to be done for development submissions, we want the file information associated with the submission in the submission file table. Since this processor is very specific to design submissions it could be reused for develop submissions after some refactoring. Alternatively this could be renamed to PostgresDesignZipMetadataProcessor, and another processor specific to develop zip could be created.
- UploadRecordProcessor
Updates values in the Upload record to be inserted into Informix. A similar process is required for development submissions. Please review the v2 api documentation and make the appropriate changes, making sure not to break design zip functionality.
- DesignZipSubmissionRecordProcessor
Updates values in the Submission record to be inserted into Informix. A similar process is required for development submissions. Please review the v2 api documentation and make the appropriate changes, making sure not to break design zip functionality.
- ResourceSubmissionRecordProcessor
Updates the resource submission record to be inserted into Informix. Please review the v2 api documentation as this table may be specific to design submissions, and potentially this is not needed for develop submisisons.
- RankProcessor
The RankProcessor is specific to the design submission process, as the way in which multiple submissions are handled is one of the main differences between design and develop submissions. This is important to understand as new functionality needs to be built to handle this correctly for develop.
In design challenges, members can submit as many entries as they want, they are simply ranked from 1 … *. Where 1 is the favorite submission in the opinion of the submitter, and n the least favorite.
In develop challenges, members can also submit as many time as they want, however only one submission remains active. Whenever a new submission is uploaded, the previous submission is marked as inactive in the upload record, and potentially the submission record as well. Please review the v2 api documentation for details on how the statuses existing submissions and upload records need to be updated when new submissions are received.
In summary, we will need a new processor to handle the status updates of existing submissions, with the specific logic for develop submissions.
- InformixPersistenceProcessor
This processor saves all the changes records to Informix. For the most part, it should be reusable as is for develop submissions. Please review the functionality against the v2 api documentation to make sure it doesn't require any tweaks.
- DesignZipCleanupProcessor
This processor is specific to design zips, and since no temporary files are needed for develop submissions, it may not be needed at all, but I am leaving this here as a reminder that any temporary files generated must be deleted.
- PostgresUpdateProcessor
This processor updates the status of the submission in Postgres to indicate the processing has ended. It can probably be used as is for develop. Please review the v2 api documentation to make sure.
This processor is also responsible for sending the success notification message to users. Please break this up, and move the success notification message to a brand new processor.
- PostProcessor
Responsible for committing the transaction and closing the db connection. Can be used as is for develop submissions.
The ProcessorFactory is also responsible for generating an error processor. The error processor is a significantly simpler chain, that performs error handling. The getErrorProcessor method needs to be updated to support the develop zip submission type, and should return just the ErrorProcessor.
- ErrorProcessor
Handles errors by updating the submission status to ERROR, rolling back the informix transaction, and sending email notifications. It can probably be used as is for develop submissions, please review the v2 api documentation to make sure.
the related source code will be attached in forum.
- Deployment Guide and Verification Steps
The submission processor consists in a chain of responsibility that is built by ProcessorFactory depending on the type of submission.
The Processor is responsible for updating legacy systems with information about the submission, so that, for instance, Online Review, Direct, and v2 api are aware of the submission. We ran an architecture challenge that describes the steps that are currently done by the v2 api to receive and process a development submissions. We need to perform all those steps as part of the development submissions feature of the processor. Please check Topcoder - Existing Development Submission Process.zip in forum for detail reference.
You need to update the ProcessorFactory to build an appropriate processor for develop submissions.
The ProcessorFactory creates a composite processor, in which each item is a processor that performs a discrete task. Some of these tasks can be reused for develop submissions, some may have to be modified.
Note, the current submission process for design submissions SHOULD NOT be impacted by these changes. If changes are made to the shared processors, please make sure to thoroughly test them with design submissions.
Existing Processors
Followings are the description for the exsting processors, suggested changes for develop submissions.- PreProcessor
Is a processor that performs some setup work for the processing of the submission. It includes creating a transaction handler the objects that represent the upload and submission records that will later be saved to Informix. This processor may be used for development submissions, but the values pre-populated onto the informix upload and submission objects must be reviewed.
- UserFetchProcessor
Simply retrieves user information, can be used as is for development submissions.
- DesignZipDownloadProcessor
This processor downloads zip files from s3 to a local folder so that they can be processed. Design submissions require changes to the zip files, which is probably not needed for develop submissions (please confirm by looking at the v2 api documentation). The develop submissions need to be copied to nfs, so a variation of this processor will be necessary to download from s3 and put the file in the correct location in the nfs share.
- DesignZipSubmissionValidatorProcessor
This processor performs validations in the data and zip files. Since the develop challenge submissions validations are significantly different, probably the best approach would be to have a separate validator processor just for develop submissions, and leave this one just for design submissions. Please review the v2 api documentation to determine which validations are done today, and perform the same validations in the submission metadata. Also please check if the zip file is valid, and not empty.
- DeclarationGeneratorProcessor, FontsProcessor, StockArtProcessor, SubmissionDeclarationFileProcessor, PreviewValidationProcessor, DesignUnifiedZipProcessor, WatermarkProcessor, and S3UploadProcessor
Not applicable to develop submissions;
- PostgresMetadataProcessor
Saves metadata about the files uploaded to Postgres. A similar process needs to be done for development submissions, we want the file information associated with the submission in the submission file table. Since this processor is very specific to design submissions it could be reused for develop submissions after some refactoring. Alternatively this could be renamed to PostgresDesignZipMetadataProcessor, and another processor specific to develop zip could be created.
- UploadRecordProcessor
Updates values in the Upload record to be inserted into Informix. A similar process is required for development submissions. Please review the v2 api documentation and make the appropriate changes, making sure not to break design zip functionality.
- DesignZipSubmissionRecordProcessor
Updates values in the Submission record to be inserted into Informix. A similar process is required for development submissions. Please review the v2 api documentation and make the appropriate changes, making sure not to break design zip functionality.
- ResourceSubmissionRecordProcessor
Updates the resource submission record to be inserted into Informix. Please review the v2 api documentation as this table may be specific to design submissions, and potentially this is not needed for develop submisisons.
- RankProcessor
The RankProcessor is specific to the design submission process, as the way in which multiple submissions are handled is one of the main differences between design and develop submissions. This is important to understand as new functionality needs to be built to handle this correctly for develop.
In design challenges, members can submit as many entries as they want, they are simply ranked from 1 … *. Where 1 is the favorite submission in the opinion of the submitter, and n the least favorite.
In develop challenges, members can also submit as many time as they want, however only one submission remains active. Whenever a new submission is uploaded, the previous submission is marked as inactive in the upload record, and potentially the submission record as well. Please review the v2 api documentation for details on how the statuses existing submissions and upload records need to be updated when new submissions are received.
In summary, we will need a new processor to handle the status updates of existing submissions, with the specific logic for develop submissions.
- InformixPersistenceProcessor
This processor saves all the changes records to Informix. For the most part, it should be reusable as is for develop submissions. Please review the functionality against the v2 api documentation to make sure it doesn't require any tweaks.
- DesignZipCleanupProcessor
This processor is specific to design zips, and since no temporary files are needed for develop submissions, it may not be needed at all, but I am leaving this here as a reminder that any temporary files generated must be deleted.
- PostgresUpdateProcessor
This processor updates the status of the submission in Postgres to indicate the processing has ended. It can probably be used as is for develop. Please review the v2 api documentation to make sure.
This processor is also responsible for sending the success notification message to users. Please break this up, and move the success notification message to a brand new processor.
- PostProcessor
Responsible for committing the transaction and closing the db connection. Can be used as is for develop submissions.
The ProcessorFactory is also responsible for generating an error processor. The error processor is a significantly simpler chain, that performs error handling. The getErrorProcessor method needs to be updated to support the develop zip submission type, and should return just the ErrorProcessor.
- ErrorProcessor
Handles errors by updating the submission status to ERROR, rolling back the informix transaction, and sending email notifications. It can probably be used as is for develop submissions, please review the v2 api documentation to make sure.
Testing Environment
Please use dev-submissions branch and follow https://github.com/appirio-tech/tc-common-tutorials/blob/dev-submissions/submission-system/README.md to setup the environment.the related source code will be attached in forum.
Final Submission Guidelines
- Code changes for the new develop submissions- Deployment Guide and Verification Steps