This topic explores a couple of methods of inserting large amounts of data into Redis.
Redis commands has a small latency, called RTT (Round Trip Time) when communicating a between the client and the server. Often, an application will send a series of small Redis commands that do not necessarily need a round-trip confirmation from the server.
$ (printf "PING\r\nPING\r\nPING\r\n"; sleep 1)|nc localhost 6379 +PONG +PONG +PONG
$ python Python 3.4.0 (v3.4.0:04f714765c13, Mar 15 2014, 23:02:41) [GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import redis >>> local_redis = redis.StrictRedis() >>> pipeline = local_redis.pipeline() >>> pipeline.ping() StrictPipeline
>> >>> pipeline.ping() StrictPipeline >> >>> pipeline.ping() StrictPipeline >> >>> pipe.execute() [True, True, True]
Open a terminal window and launch redis-cliand check to see that Redis is empty.
127.0.0.1:6379> DBSIZE (integer) 0 127.0.0.1:6379> exit
Next, download the text file high-temp-2014.txt, that contains the daily high temperature in 2014 for San Jose. we
$ (cat high-temp-2014.txt; sleep 10) | nc localhost 6379 +OK +OK . . . +OK
Now launch redis-cli and check to see the number of keys in Redis with DBSIZE command and retrieve some sample keys with GET.
127.0.0.1:6379> DBSIZE (integer) 364 127.0.0.1:6379> DBSIZE (integer) 364 127.0.0.1:6379> GET 2014-1-1 "62" 127.0.0.1:6379> GET 2014-7-21 "79" 127.0.0.1:6379> GET 2014-12-31 "55"
In redis-cli, flush the datastore with FLUSHALL to start with an empty datastore.
127.0.0.1:6379> FLUSHALL OK 127.0.0.1:6379> exit
Run a python script that inserts inserts 10,000 strings into Redis.
$ cd ~/introduction-to-redis/pipelining-and-mass-insertions $ python protocol.py | ~/redis-3.0.2/src/redis-cli --pipe All data transferred. Waiting for the last reply... Last reply received from server. errors: 0, replies: 10000
Finally, using redis-cli, check the size with the DBSIZE command and retrieve some sample keys with GET.
~/redis-2.8.18/src/redis-cli 127.0.0.1:6379> DBSIZE (integer) 10000 127.0.0.1:6379> GET Key:1 "Value=1" 127.0.0.1:6379> GET Key:4567 "Value=4567" 127.0.0.1:6379> GET Key:9000 "Value=9000"