Challenge Overview
In this challenge series we're building a microservice for integration purposes that listens for incoming files over SFTP and streams the output on a Kafka bus. Service will be based on Apache Camel.
So far we built an SFTP server that listens for incoming files, forwards them into the Camel pipeline that braks the input file on newlines and pushes each line as a message on Kafka. One specific feature that we want to support is handling large files (up to 500GB) without the need to store them in memory or on disk. Our SFTP server supports that use case with help of the file-systems FIFO stream. The Apache Camel pipeline is instructed to listen to file creations and the FIFO is processed, but when closed by both ends will be cleared of bufferdata(RAM memory). This approach also requires the deletion of FIFO files to occur when Apache Camel pipeline processing has completed.
NOTE: There is no frontend interface to test here, only service functionality.
In this challenge we want to perform testing on the microservice and find any issues with the above workflow. Any issues that break the above workflow will be considered valid bugs. Areas that we would like to cover in this bug hunt are:
- Test for any scenarios that cause a crash or unhandled errors
- Testing with large files (>1GB)
- Testing multiple concurrent file uploads (are there any concurrency issues, race conditions, etc)
- Testing long input lines
NOTE: Bug hunt will start on 05/13 06:00AM EDT
Microservice code will be posted in the forums (once the bug hunt starts) and it contains build/run/deploy steps using docker containers.
All issues should be created on the project repository available here.
Payment Structure and Scoring
The participant with the highest number of valid bugs (reproducible by reviewer by following steps & video/ screenshot) logged will be declared the winner and will earn the first place payment
For other participants, we will pay up to $40 ($40 - critical, $20 - major, $5 - minor/cosmetic/ enhancement) for each unique bug logged. The total amount paid will not exceed half of first place prize.