Spring Batch tip – Partitioning

Basically, Spring Batch Job runs in single thread. To increase the throughput, we need to parallelize the job by partitioning. The core architecture of Job partitioning is as follows. Partition Step : The wrapper of common Step with a partitioner and grid size (or partition size) Single Step : Common Step with a Reader, a …

Spring Batch – Read once, Write multi example

Sometimes, we need to read once, write multi ETL with Spring Batch. Spring Batch supports it with CompositeItemWriter (org.springframework.batch.item.support.CompositeItemWriter). Sample table schema Declaration of CompositeItemWriter Make sure that CompositeItemWriter has 2 delegates (TestWriter1, TestWriter2). You can use any Writer as it's delegate. Full Job declaration Make sure that Job task is using new CompositeItemWriter instead …

More on Spring Batch Writer

Spring Batch Writer is the implementation of org.springframework.batch.item.ItemWriter. You can write a custom writer, but Spring Batch already has some useful implementations, such as org.springframework.batch.item.amqp.AmqpItemWriter for AMQP brokers such as RabbitMQ org.springframework.batch.item.file.FlatFileItemWriter for a file org.springframework.batch.item.database.JpaItemWriter for database using JPA org.springframework.batch.item.database.JdbcBatchItemWriter for writing to database with plain SQL This post shows detailed information on using …

More on Spring Batch Reader

Spring Batch Job is composed of Reader, Processor and Writer. This post shows some more details on Reader. Reader is the implementation of org.springframework.batch.item.Reader. I recommend to use the following implementations. AmqpItemReader (org.springframework.batch.item.amqp.AmqpItemReader) : to interact with queue such as RabbitMQ FlatFileItemReader (org.springframework.batch.item.file.FlatFileItemReader) : to interact with a file JdbcCursorItemReader (org.springframework.batch.item.database.JdbcCursorItemReader) : to interact with …

Simple ETL with Spring Batch

Spring Batch is literally a batch framework based on Spring Framework. I usually use it to develop a simple ETL(Extraction, Transaformation and Loading) program. In this post, I'll show you how to write a simple ETL program. (This sample is tested on Spring Batch 3.0.10) Prerequisites Database (MySQL or Oracle) Spring batch context database Spring …