http://intro2libsys.info/pycon-jp-2015

Redis

Redis is a NoSQL key-value data server, recently ranked as one of the top technologies for data representation. Going beyond a simple string key and value store like Memecache, Redis provides a rich set of data structures like hashes, lists, sets, sorted-sets, and most recently a GIS data structures that stores all of the data in a machines RAM while providing two types of persistence; a snapshot variant that periodically saves everything to disk and an append-only mode, that logs every change to disk.

Redis Overview

Redis is an open-source, written in C, with a design goal to extremely fast and memory efficient as all of the data is loaded in RAM.

The official documentation of Redis is extensive and as a bonus of providing you the Big O notation about the efficiency of your Redis commands for specific types of data structures. In this way, you can do an quick estimate of the time efficiency of your data structures in your code.

O(1) == Good!!

String and Hash Example
>>> import redis
>>> local_redis = redis.StrictRedis()
>>> local_redis.set("ichi", 1)
True
>>> local_redis.get("ichi")
b'1'
>>> local_redis.hmset("numbers", {"ichi":1, "ni":2, "san": 3})
>>> local_redis.hgetall("numbers")
{b'ni': b'2', b'san': b'3', b'ichi': b'1'}
>>> local_redis.hget("numbers", "san")
b'3'  
    

Lua Server-side Functions

Redis's design pattern for loading and executing server-side Lua scripts was an inspiration for the design for the Linked Data Fragments Server. For these Server Side scripts, you can load the script once and a SHA1 hash digest of the script is returned that you can use in subsequent function calls instead of sending the entire script over the network each time you want to execute the function.

Example
>>> local_redis = redis.StrictRedis()
>>> lua_script = """redis.pcall('incr', 'counter')"""
>>> count_sha1 = local_redis.script_load(lua_script)
>>> count_sha1
'cfb194a8e050ff9f25dc208005798133b5630b51'
>>> local_redis.get('counter')
>>> local_redis.evalsha(count_sha1, 0)
>>> local_redis.get('counter')
b'1'
    

(Sorta) Unrelated Aside

I'm currently writing a book, Mastering Redis for Packt Publishing due out in 2016 that goes into much more detail on using Redis for application development and operations.

Linked Data Fragments
in Redis

For each subject, predicate, and object in a RDF graph's triple, a SHA1 hash digest is calculated from the string serialization of the each element and used as a key with the actual entity's value stored at that location.

Original Value SHA1
http://catalog.coloradocollege.edu/4124646 8b55509dc25092e441bcecb81312f281df945562
http://www.w3.org/1999/02/22-rdf-syntax-ns#type 3c197cb1f6842dc41aa48dc8b9032284bcf39a27
http://bibframe.org/vocab/Work 5d1377f4476a1cbfb3caea106dc6b0a7d086410a
>>> local_redis.set(
	'5d1377f4476a1cbfb3caea106dc6b0a7d086410a',
	'http://bibframe.org/vocab/Work')
True
>>> local_redis.get(
	'5d1377f4476a1cbfb3caea106dc6b0a7d086410a')
b'http://bibframe.org/vocab/Work'
    

For each triple in the RDF graph, a composite key is constructed with the subject SHA1 hash digest, predicate SHA1 hash digest, and object SHA1 hash digest separated by a colon ":".

To store the triple above, we set an integer value of 1 for the key:
8b55509dc25092e441bcecb81312f281df945562:3c197cb1f6842dc41aa48dc8b9032284bcf39a27:5d1377f4476a1cbfb3caea106dc6b0a7d086410a

About RDF & Linked Data BIBFRAME & DPLA Redis Python Falcon & Asynco Experiments Next Steps Resources