Challenge Overview
In this challenge series we're building a microservice for integration purposes that listens for incoming files over SFTP, splits the contents into smaller messages and streams the output on a Kafka bus. Service will be based on Apache Camel.
In this challenge, we want to add business logic to the processor - splitting the records into manageable parts and forwarding individual messages to Kafka.
Base code for this project is available in the forums and it contains a preconfigured Camel pipeline and application build/run/deploy steps using docker containers.
Splitting the records will be a simple task - we want to start with splitting the input on newline (so each line would end up as a separate message in Kafka). Input processor should read the input message as a stream (entire message will be too big to fit into memory). Make sure to properly close the stream, clean up any used resources and handle errors in reading from the stream, processing the message and sending the output to Camel exchange.
Submit a short deployment/verification guide
In this challenge, we want to add business logic to the processor - splitting the records into manageable parts and forwarding individual messages to Kafka.
Base code for this project is available in the forums and it contains a preconfigured Camel pipeline and application build/run/deploy steps using docker containers.
Splitting the records will be a simple task - we want to start with splitting the input on newline (so each line would end up as a separate message in Kafka). Input processor should read the input message as a stream (entire message will be too big to fit into memory). Make sure to properly close the stream, clean up any used resources and handle errors in reading from the stream, processing the message and sending the output to Camel exchange.
Final Submission Guidelines
Submit the updated codeSubmit a short deployment/verification guide