MySQL gap lock, next key lock by example

MySQL's gap lock or next key lock is a specific mechanism to guarantee it's isolation level. Gap lock is set on before or after values of the accessed record. Next key lock is the accessed row lock + gap lock. Therefore they have similar features. Gap lock is activated in the following conditions. Isolation level … Continue reading MySQL gap lock, next key lock by example →

The difference of Isolation level – Read Committed vs Repeatable Read

MySQL's default isolation level is "Repeatable Read". But Oracle's default isolation level is "Read Committed". In this post, I will show you the difference by example. Test environment : MySQL 5.7 Test table Test scenario 1 - Read Committed Seq Session 1 Session 2 1 set session transaction isolation level read committed; start transaction; set … Continue reading The difference of Isolation level – Read Committed vs Repeatable Read →

Kafka tip – keeping the order of delivery

Kafka guarantees the order of delivery within a partition. That is, the order is not kept among partitions in a topic. There are some tips to keep the order of delivery in a topic. 1. One partition for a Topic This is what Kafka basically supports. <Pros> Simple solution <Cons> A partition is applied to … Continue reading Kafka tip – keeping the order of delivery →

Spring Batch tip – Partitioning

Basically, Spring Batch Job runs in single thread. To increase the throughput, we need to parallelize the job by partitioning. The core architecture of Job partitioning is as follows. Partition Step : The wrapper of common Step with a partitioner and grid size (or partition size) Single Step : Common Step with a Reader, a … Continue reading Spring Batch tip – Partitioning →

Spring Batch – Read once, Write multi example

Sometimes, we need to read once, write multi ETL with Spring Batch. Spring Batch supports it with CompositeItemWriter (org.springframework.batch.item.support.CompositeItemWriter). Sample table schema Declaration of CompositeItemWriter Make sure that CompositeItemWriter has 2 delegates (TestWriter1, TestWriter2). You can use any Writer as it's delegate. Full Job declaration Make sure that Job task is using new CompositeItemWriter instead … Continue reading Spring Batch – Read once, Write multi example →

More on Spring Batch Writer

Spring Batch Writer is the implementation of org.springframework.batch.item.ItemWriter. You can write a custom writer, but Spring Batch already has some useful implementations, such as org.springframework.batch.item.amqp.AmqpItemWriter for AMQP brokers such as RabbitMQ org.springframework.batch.item.file.FlatFileItemWriter for a file org.springframework.batch.item.database.JpaItemWriter for database using JPA org.springframework.batch.item.database.JdbcBatchItemWriter for writing to database with plain SQL This post shows detailed information on using … Continue reading More on Spring Batch Writer →

Simple ETL with Spring Batch

Spring Batch is literally a batch framework based on Spring Framework. I usually use it to develop a simple ETL(Extraction, Transaformation and Loading) program. In this post, I'll show you how to write a simple ETL program. (This sample is tested on Spring Batch 3.0.10) Prerequisites Database (MySQL or Oracle) Spring batch context database Spring … Continue reading Simple ETL with Spring Batch →

A tip on using user transaction on Mybatis

When using transation on Mybatis, there are 3 ways. Container managed transaction - JEE engine manages transaction User managed transaction - transaction is controlled by user source code Spring managed transaction - transaction is controlled by Spring Transaction policy I usually use Spring managed or container managed transaction. But sometimes, I need to control transaction … Continue reading A tip on using user transaction on Mybatis →

Some useful commands to manage Kafka

This post shows some useful commands to manage Kafka. (based on version 2.12) Detailed information is on Kafka official documentation Notice that some commands use zookeeper ip/port and others use kafka ip/port Topic management commands List all topics ${KAFKA_HOME}/bin/kafka-topics.sh --list --zookeeper zookeeper_ip:port Create a topic ${KAFKA_HOME}/bin/kafka-topics.sh --create --zookeeper zookeeper_ip:port --replication-factor num --partitions num --topic topic_name … Continue reading Some useful commands to manage Kafka →

Author: TK