Spring Batch tip – Partitioning

Basically, Spring Batch Job runs in single thread. To increase the throughput, we need to parallelize the job by partitioning. The core architecture of Job partitioning is as follows. Partition Step : The wrapper of common Step with a partitioner and grid size (or partition size) Single Step : Common Step with a Reader, a … Continue reading Spring Batch tip – Partitioning

Spring Batch – Read once, Write multi example

Sometimes, we need to read once, write multi ETL with Spring Batch. Spring Batch supports it with CompositeItemWriter (org.springframework.batch.item.support.CompositeItemWriter). Sample table schema Declaration of CompositeItemWriter Make sure that CompositeItemWriter has 2 delegates (TestWriter1, TestWriter2). You can use any Writer as it's delegate. Full Job declaration Make sure that Job task is using new CompositeItemWriter instead … Continue reading Spring Batch – Read once, Write multi example

More on Spring Batch Writer

Spring Batch Writer is the implementation of org.springframework.batch.item.ItemWriter. You can write a custom writer, but Spring Batch already has some useful implementations, such as org.springframework.batch.item.amqp.AmqpItemWriter for AMQP brokers such as RabbitMQ org.springframework.batch.item.file.FlatFileItemWriter for a file org.springframework.batch.item.database.JpaItemWriter for database using JPA org.springframework.batch.item.database.JdbcBatchItemWriter for writing to database with plain SQL This post shows detailed information on using … Continue reading More on Spring Batch Writer

More on Spring Batch Reader

Spring Batch Job is composed of Reader, Processor and Writer. This post shows some more details on Reader. Reader is the implementation of org.springframework.batch.item.Reader. I recommend to use the following implementations. AmqpItemReader (org.springframework.batch.item.amqp.AmqpItemReader) : to interact with queue such as RabbitMQ FlatFileItemReader (org.springframework.batch.item.file.FlatFileItemReader) : to interact with a file JdbcCursorItemReader (org.springframework.batch.item.database.JdbcCursorItemReader) : to interact with … Continue reading More on Spring Batch Reader