[96hr] Topcoder Project Service - Link Attachments and Tags

Update attachments functionality to support links and tags.

Project Background

Topcoder Project Service is the main backend service of Topcode Connect – the client-facing application of Topcoder. We have the functionality to add “files” and “links” to the projects.

• To store and manage “files” we have a ParojectAttachment model with relative endpoints. As attachments are stored as objects we can track who creates and updates each individual attachment and we can restrict the visibility of attachments for particular users using `allowedUsers` field. 

• Unlike “files” we store “links” inside “bookmarks” filed of the “project” object so we cannot track who creates and updates individual links properly and we also cannot restrict access to the individual links as all the links are included in the project object and always updated and returned altogether.

We would like to have the same features for “links” which we have for “files”. So we want to update the ParojectAttachment model with related endpoints to also support “links” as attachments. Additionally, we would like to be able to add tags to attachments (files and links). 

Technology Stack

• Node.js

• PostgreSQL

• Elasticsearch

Code access

The work for this challenge has to be done in 2 repositories:
- Project Service repo https://github.com/topcoder-platform/tc-project-service branch `feature/link-attachments` commit `b2dbc560fbe7f83789bd1317fa562db9a26980eb` or later.
- Project ES Processor repo https://github.com/topcoder-platform/project-processor-es branch `feature/link-attachments` commit `5fa17456bbf646039b8b6f64fc49bcd96e490afe` or later.

- As code in 2 repositories has to be updated, please, use the “Local Setup with Multiple Services” guide for setting up the project locally. Steps 5 and 6 in the guide are not necessary for this challenge, so you may ignore them.

- Config is provided on the forum.

Individual requirements

1) Update model and endpoints

Update all the code related to attachments to support “links” and “tags”:

• Update ProjectAttachment model
  Add fields:
  - “type” - String, required, allowed values: “file” and “link” - determines if the attachment is link or file
  - “tags” - JSON, required, default “[]” - JSON array of string tags, example `[“specification”, “design preview”, “billing information”]`
  - Rename field “filePath” to “path” (this would require to update it in all the places across the codebase). We would use this field for both: to keep link URL and to keep file path as we do now.

• Create SQL migration script in “/migrations” folder to update DB accordingly to model changes.

• Update all the attachment endpoints Joi validation rules according to model changes.

• Make endpoint `GET /v5/projects/:projectId(\\d+)/attachments/:id(\\d+)` more general and suitable for both “files” and “links”:
  - Rename “download.js” to “get.js”.
  - Instead of an object with only one “url” property, it should return the whole ParojectAttachment object.
  - For “links”, this object would be returned as it is.
  - For “files”, we should return ParojectAttachment object with an additional “url” field with the presigned URL as it’s already done now.

• Update ElasticSearch mapping for ES document “project” according to ParojectAttachment model changes (in Project Service and Project ES Processor) see here and there.

• Update “project-processor-es” Kafka message handlers to validate ParojectAttachment objects according to changes.

• Update “project-processor-es” unit tests, so example data in unit tests have some data in all the new fields and make sure that unit tests pass.

2) Kafka messages

We are sending some Kafka messages when we are doing operations with “links” and “files”. Such messages may be consumed by other services to make some actions. For example, TC Notification Services creates notifications to be shown to users when files or links are created.

As links would be no more stored in “project.bookmarks” but would be stored in ParojectAttachment model together with files, please, make sure that existent Kafka messages and related functionality still works correctly after changes. In particular:

• make sure that “CONNECT_NOTIFICATION_EVENT.PROJECT_FILE_UPLOADED” is only produced for attachments with `type=“files”` but not for “links”

• make sure that Kafka message “CONNECT_NOTIFICATION_EVENT.PROJECT_LINK_CREATED” is sent when the “link” is added as an attachment

• rename Kafka topic “connect.notification.project.files.updated” to “connect.notification.project.attachment.updated” everywhere together with the relative constant “PROJECT_FILES_UPDATED” -> “PROJECT_ATTACHMENT_UPDATED” to reflect the new meaning of this event.

• update/write unit tests which would check that all required Kafka messages are sent for “links” and for “files”

3) Links data migration scripts (two ways)

3.1) Create a script for migrating existent links from “project.bookmarks” to the attachment model. See example project data with bookmarks.

• Put the script inside “/migrations” folder. 

• The script should get all required config values from the environment variables.

• For each link object inside the “bookmarks” array for each project in DB script should create a “projectAttachment” object with:
  - “type”: “link” (always for links)
  - "title": from “bookmark.title”
  - “path”: from “bookmark.address”
  - “createdAt”: from “bookmark.createdAt”
  - “createdBy”: from “bookmark.createdBy”
  - “updatedAt”: from “bookmark.updatedAt”
  - “updatedBy”: from “bookmark.updatedBy”
  - “tags”: []
  - "size", "category", “description”, “filePath”, “contentType”, “allowedUsers”: set null
  - if some link doesn’t have “createdAt”, “createdBy”, “updatedAt”, “updatedBy” take them from project object where the link belongs too.

3.2) Make the second script to make the backward migration of links from “projectAttachment” model to “project.bookmarks”. We may use this script if something goes wrong after migrating.

Note, that script should only update data in the DB, we would reindex all the projects in the Elasticeasrch index after migration.

General requirements

• We don’t require coverage percentage for unit tests, but, please, create unit tests to cover requirements in this challenge where possible.

• Keep the existent logic with “bookmarks” as it is for now, so we may fall back to it if something goes wrong with migration to the new model.

• Update the swagger file accordingly to the changes.

• Update the postman file to have an example of creating a link as an attachment.

• Adjust unit tests in Project Service repo according to the changes. If you think that some unit test doesn’t have sense anymore after changes, please confirm on the forum before removing.

• LInt should pass (don’t disable lint rules).

• All unit tests should pass.

Validation guide

To validate the data migration script during the review, please, provide a DB dump with some demo data that would show different cases. At least 3 projects and 10 links. 

Also, provide a quick validation guide on how to apply such a DB dump and how to run a migration script both ways.

Final Submission Guidelines

Submit a zip file which would include:

• 2 git patches with changes you’ve made to the code in our repositories

• Validation guide in any textual format

• DB dump(s) with demo data

Additionally, the winner would be required to raise pull requests to the repositories after the challenge is completed.