The use of Zookeeper

Zookeeper was developped to support Hadoop cluster. Afterward, it has become an independent cluster management solution, although it is still used as Hadoop cluster manager.

I used Zookeeper as a cache cluster management solution which has nothing to do with Hadoop. Now, I’m showing you some use of Zookeeper.

Basic knowledge

Zookeeper keeps internal data structure which is called “znode”. Znodes are in shape of directory. For example, as follows.

For the above struture, node_11 is access as /node_1/node_11.

There are 4 types of znode. (based on 3.4.x)

• persistent : When a znode is created, it persists until deleted explicitly
• ephemeral : When a znode is created, it is deleted automatically when the client which created it disconnects
• persistent sequential : Same with persistent node except that a new node is suffixed with incremental number
• ephemeral sequential : Same with ephemeral node except that a new node is suffixed with incremental number

Client can create a znode, delete it, set data of it or check existence of it.

Client also can register a Watcher. Watcher’s role is to get notified on znode events. (Event types : NodeCreated, NodeDeleted, NodeDataChanged, NodeChildrenChanged) Using a Watcher, a client can get a node’s change in real time without polling Zookeeper frequently.

Use case : Master – Slave management

I think it’s the most common use case. In cluster, it is very important which node is a master. When a node is down, another node must take over the role. For some clusterware, every node exchanges heart beat and vote for a new master node when an active node is down.

But with Zookeeper, the logic changes in simple way.

1. A node creates an ephemeral znode. If another node tries to create a znode with the same name, it fails. This type of transaction is guaranteed by Zookeeper. Now, the znode owner becomes an active node
2. The other nodes (including failed node) wait in slave mode. And they register a Watcher for the znode existance
3. If current master dies, Zookeeper notfies all the waiting slaves.(NodeDeleted event) As soon as a slave succeeds to create a znode, it becomes a new master

Use case : Managing cluster meta info

In cluster, some meta data needs to be saved and retrieved by cluster members. But if the repository is down, all members can have problem in working. The crucial factor here is that 1)  meta data repository must be available 24*365 2) if there are multi repositories, each one must have consistent image.

Zookeeper ensemble (Zookeeper cluster) meets this requirement. If a Zookeeper node gets some change request, it replicates the request to all the other Zookeeper nodes. Therefore, one Zookeeper’s down don’t affect the cluster.

Let me introduce my case.

• There is a cache cluster with multiple Redis processes.
• Each WAS shards cache data based on Redis count.
• But if a Redis were down, each WAS could have different count (i.e. WAS#1 knows the Redis’s down, but WAS#2 don’t know it)
• So I saved Redis alive info on Zookeeper ensemble.
• A checker checks Redis status periodiacally and updates the info.
• Each WAS registers a Watcher and get notified when Zookeeper info changes. Then, all sharding calculation changes concurrently