Introduction to Redis

Redis Cluster

Redis Cluster provides one method of running Redis where data is sharded across multiple Redis instances while providing a degree of availability for continuing operations in the event of a node failing or a communication issue.

Nodes in the Redis Cluster use the Redis cluster protocol to connect with every other node in the Redis cluster for a mesh network topology. Nodes communicate using a TCP gossip protocol and along with a configuration update mechanism in order to reduce the number of messages being exchanged between nodes.

Composite Partitioning

Redis cluster uses a form of composite partitioning called consistent hashing that calculates what Redis instance any particular key is assigned called a hash slot.

The hash slot is the CRC16 hash algorithm applied to the key and then the computation of a modulo using 16384. The specific algorithm used by Redis cluster to calculate the hash slot for a key:

  1. The cyclic redundancy check (CRC) using a polynomial length of 17-bits or CRC16 is calculated on the Redis key
  2. The modulo 16384 of the value is then calculated to get the hash slot for the key.

Each master node is assigned a sub-range of hash slots and the key and value will reside on that Redis instance.

When a Redis cluster is running, each node has two TCP sockets open:
  • First is the standard Redis protocol for connecting clients (default port 6379 for the first node)
  • Second port is calculated from the sum of the first port plus 10000 (16379 for the default port) and binary protocol for node-to-node communication.
Clients should never need to connect directly with the cluster bus port but with the lower, standard port.

Six-Node Redis Cluster

To illustrate all of the features of Redis Cluster, the minimum recommended Redis cluster setup is to have a 3 Master Redis nodes with each master node replicated with a single Redis slave instance node. Each Master Node hash slots are broken down as

Master One (M1) - allocated hash slots 0-5460
Slave One (S1) replicates Master One, is promoted if a quorum of nodes cannot reach Master One.
Master Two (M2) - allocated hash slots 5461-10922
Slave Two (S2) replicates Master Two, is promoted if a quorum of nodes cannot reach Master Two.
Master Three (M3) - allocated hash slots 10923-16383
Slave Three (S3) replicates Master Three, is promoted if a quorum of nodes cannot reach Master Three.

Exercise: Configuring and Running Redis Cluster

We'll start by manually create a minimal cluster with three master nodes and three slaves.

$ cd introduction-to-redis/clustering-and-ha/
$ mkdir cluster-test
$ cd cluster-test
$ mkdir 7000 7001 7002 7003 7004 7005
$ touch 7000/redis.conf
$ vi 7000/redis.conf
$ cp 7000/redis.conf 7001/.
$ vi 7001/redis.conf
$ cp ~/redis-3.0.2/src/redis-server .
$ cd 7000
$ ../redis-server redis.conf
# New terminal tab
$ cd ../7001
$ ../redis-server redis.conf
$ ../redis-server redis.conf


Install the Ruby Redis gem and run the redis-trib.rb

$ sudo gem install redis
$ cd ../../
$ .~/redis-3.0.2/src/redis-trib.rb create --replicas 1 \

Sample redis.conf

Here is the example redis.conf we are using for each of Redis cluster nodes

port 7000
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
appendonly yes

We need to change the port number to the correct value for the node, i.e. for node 7002, we need specify the correct port number for that node's redis.conf configuration file.

Exercise: Using Redis Cluster

After successfully running redis-trib.rb, you can interact with the cluster using redis-cli

$ ~/redis-3.0.2/src/redis-cli -c -p 7000> SET Book:1 "Moby Dick"
-> Redirected to slot [14335] located at
OK> SET Book:2 "Infinite Jest"
-> Redirected to slot [1948] located at
OK> GET Book:1
-> Redirected to slot [14335] located at
"Gone with the Wind"> GET Book:2
-> Redirected to slot [1948] located at
"Infinite Jest"> 

Redis commands that involve multiple keys, like MSET will not work with redis cluster> MSET Book:3 "Moby Dick" Book:4 "Pride and Prejudice" Book:5 "Tom Sawyer"
(error) CROSSSLOT Keys in request don't hash to the same slot

Easier Redis Cluster with create-cluster script

Instead of manually setting up and running a Redis cluster, you can instead use the create-cluster located in the redis/utils/create-cluster.

Starting in the create-cluster directory, follow these steps to get a 6-node Redis Cluster with 3 masters and 3 slaves.

  1. Create a script:
  2. Start Redis Cluster:
    ~/redis/utils/create-cluster$ ./create-cluster start
    Starting 7001
    Starting 7002
    Starting 7003
    Starting 7004
    Starting 7005
    Starting 7006
  3. Link and create the Redis Cluster
  4. >>> Creating cluster
    >>> Performing hash slots allocation on 6 nodes...
    Using 3 masters:
    Adding replica to
    Adding replica to
    Adding replica to
    M: 1379949e7d8eaa27a0634285e521079eccc0cc1f
       slots:0-5460 (5461 slots) master
    M: 470bf4397e0002f211df09dadcd5ec12b458e9c3
       slots:5461-10922 (5462 slots) master
    M: 7e343391d165ccee34e0f1cf43590270130a9d5b
       slots:10923-16383 (5461 slots) master
    S: 3d8c532367f0846f292b538d09b7cafdc6b3c6b9
       replicates 1379949e7d8eaa27a0634285e521079eccc0cc1f
    S: 491abeb14973c0c9495f1b045b4e5d3f0729bcc8
       replicates 470bf4397e0002f211df09dadcd5ec12b458e9c3
    S: 933258e1d5ed8752c7e4ff7ce377dfd63543977f
       replicates 7e343391d165ccee34e0f1cf43590270130a9d5b
    Can I set the above configuration? (type 'yes' to accept): yes
    >>> Nodes configuration updated
    >>> Assign a different config epoch to each node
    >>> Sending CLUSTER MEET messages to join the cluster
    Waiting for the cluster to join..
    >>> Performing Cluster Check (using node
    M: 1379949e7d8eaa27a0634285e521079eccc0cc1f
       slots:0-5460 (5461 slots) master
    M: 470bf4397e0002f211df09dadcd5ec12b458e9c3
       slots:5461-10922 (5462 slots) master
    M: 7e343391d165ccee34e0f1cf43590270130a9d5b
       slots:10923-16383 (5461 slots) master
    M: 3d8c532367f0846f292b538d09b7cafdc6b3c6b9
       slots: (0 slots) master
       replicates 1379949e7d8eaa27a0634285e521079eccc0cc1f
    M: 491abeb14973c0c9495f1b045b4e5d3f0729bcc8
       slots: (0 slots) master
       replicates 470bf4397e0002f211df09dadcd5ec12b458e9c3
    M: 933258e1d5ed8752c7e4ff7ce377dfd63543977f
       slots: (0 slots) master
       replicates 7e343391d165ccee34e0f1cf43590270130a9d5b
    [OK] All nodes agree about slots configuration.
    >>> Check for open slots...
    >>> Check slots coverage...
    [OK] All 16384 slots covered.
  5. Stopping Redis Cluster
    $ ./create-cluster stop
    Stopping 7001
    Stopping 7002
    Stopping 7003
    Stopping 7004
    Stopping 7005
    Stopping 7006

Redis Sentinel

Redis Sentinel manages Redis instances through four main ways:

  • Monitors running Redis masters and slaves
  • Notifies system administrators or monitoring software of problems with Redis instances
  • Automatic failover by promoting running slaves to master if a master instance is experiencing problems
  • Configuration provider for clients to connect and discover the address for a given Redis master.

Exercise: Configuring and Running Redis Sentinel

Snippet from sentinel.conf

# dir 
dir /tmp

# sentinel monitor    
sentinel monitor mymaster 6379 2

# Default is 30 seconds.
sentinel down-after-milliseconds mymaster 30000

Running Redis Sentinel can be accomplished either with the redis-sentinel binary or by passing the --sentinel parameter when running redis-server. Both options require a sentinel.conf file.

$ redis-sentinel /path/to/sentinel.conf
$ redis-server /path/to/sentinel.conf --sentinel

You can connect directly to the sentinel with redis-cli and query to find the status of monitored Redis instances

$ ../redis-3.0.2/src/redis-cli -p 26379> PING
PONG> SENTINEL masters> SENTINEL slaves mymaster

References and Resources

  1. From the website: Redis cluster tutorial
  2. redis-py-cluster
  3. Redis Sentinel Documentation