Redis architecture for production

Redis is a very fast cache server. But in production, there is something to consider.

Redis works in single thread

To be exact, Redis is a multi threaded server. But the main logic runs in a single thread.

For the following single Redis process

[tkstone@localhost bin]$ ps -p 23073 -f
UID        PID  PPID  C STIME TTY          TIME CMD
tkstone  23073     1  0 Nov22 ?        00:09:36 /home/tkstone/redis-4.0.6/bin/redis-server *:6379

when I check it’s threads

[tkstone@localhost bin]$ ps -p 23073 -L -f
UID        PID  PPID   LWP  C NLWP STIME TTY          TIME CMD
tkstone  23073     1 23073  0    4 Nov22 ?        00:09:36 /home/tkstone/redis-4.0.6/bin/redis-server *:6379
tkstone  23073     1 23077  0    4 Nov22 ?        00:00:00 /home/tkstone/redis-4.0.6/bin/redis-server *:6379
tkstone  23073     1 23078  0    4 Nov22 ?        00:00:00 /home/tkstone/redis-4.0.6/bin/redis-server *:6379
tkstone  23073     1 23079  0    4 Nov22 ?        00:00:00 /home/tkstone/redis-4.0.6/bin/redis-server *:6379

The result shows that a Redis process is composed of 4 threads.

• Thread 1 : main thread which works for servicing almost every command
• Thread 2 : AOF file close thread. It is active even when AOF is disabled
• Thread 3 : AOF write thread which runs every 1 second. It’s enabled when AOF is disabled
• Thread 4 : UNLINK, FLUSHALL or FLUSHDB command processor. (Those commands run asynchronosly) Added from version 4

Therefore, one Redis process usually use 1 cpu core. (by thread 1)

Recommended architecture for Redis

To make full use of a production system, I recommend the following architecture.

• Run multi Redis processes per a machine (process count should be equal to that of cpu core)
• For HA, run slave Redis on another machine
• Slave machine should have the same computing power with the master
• For automatic failover, run several Sentinels (each Sentinel should run on a separate machine.) To avoid split brain issue, odd number is recommended
• Sentinel is a lightweight process. Therefore Sentinel server don’t need high computing power
• To distribute traffic to multi Redis processes evenly, client needs to implement data sharding

More on data sharding

• Redis is a key-value store
• Sharding rule based on key is recommended
• For the same key, sharding target must be consistent
• All clients must have the same sharding rule
• Some client library (i.e. Jedis) supports data sharding
• When data sharding is enabled, secondary index should be avoided. (More on secondary index)

Limitation on the above architecture

• If a master and the matching slave die, 1/n of data is lost
• Sentinel supporting client library is needed. From my experience, Redis and Lettuce support Sentinel
• Supporting Sentinel and data sharding together makes client development difficult. Next time, I’m explaining how to do it with Jedis