Introduction

I have been doing a lot of work with Redis in the past twelve months, using it for cache, session, and messaging data. It is a fast and reliable system. A fourth area I wanted to tackle however was long-term storage. Redis is somewhat limited as a candidate for this by the fact that you need to keep all of your data in main memory, so you really have to watch your data growth with it, but regardless I still wanted to explore the possibility of using it for storing business objects from the Alpha Framework, which is classical tabular data presently stored in either MySQL or SQLite depending on the provider configured.

In Alpha, the object storage providers implement the AlphaDAOProviderInterface, which is quite mature and well tested at this stage. Here are my notes from assessing Redis as a potential provider of that interface.

Storing a table of data

Start with a typical business object list like this:

OID	firstname	lastname	email	notes
1	John	Doe	john@nowhere.com	Some person I know
2	Jack	Doe	jack@nowhere.com	Another person I know
3	Jock	Doe	jock@nowhere.com	Yet another person I know

Now we will store these in Redis as a hashes:

redis 127.0.0.1:6379> hmset ContactObject-1 OID 1 firstname "John" lastname "Doe" email "john@nowhere.com" notes "Some person I know"
redis 127.0.0.1:6379> hmset ContactObject-2 OID 2 firstname "Jack" lastname "Doe" email "jack@nowhere.com" notes "Another person I know"
redis 127.0.0.1:6379> hmset ContactObject-3 OID 3 firstname "Jock" lastname "Doe" email "jock@nowhere.com" notes "Yet another person I know"

To retrieve one of those objects:

redis 127.0.0.1:6379> hgetall ContactObject-2
 1) "OID"
 2) "2"
 3) "firstname"
 4) "Jack"
 5) "lastname"
 6) "Doe"
 7) "email"
 8) "jack@nowhere.com"
 9) "notes"
10) "Another person I know"

Now add each of those to a ContactObjects set:

redis 127.0.0.1:6379> sadd ContactObjects "ContactObject-1"
(integer) 1
redis 127.0.0.1:6379> sadd ContactObjects "ContactObject-2"
(integer) 1
redis 127.0.0.1:6379> sadd ContactObjects "ContactObject-3"
(integer) 1
redis 127.0.0.1:6379> smembers ContactObjects
1) "ContactObject-3"
2) "ContactObject-1"
3) "ContactObject-2"

That set is effectively our master set of ContactObjects.

What about incrementing those OIDs? From your application, call this before prefixing the result with "ContactObject-":

redis 127.0.0.1:6379> incr ContactObjectsMaxOID
(integer) 1
redis 127.0.0.1:6379> incr ContactObjectsMaxOID
(integer) 2
redis 127.0.0.1:6379> incr ContactObjectsMaxOID
(integer) 3

What if we want to maintain a relation of what contacts belong to a given user? For this we will need to use another set:

127.0.0.1:6379> sadd UserObject-1-ContactObjects "ContactObject-1"
(integer) 0
127.0.0.1:6379> sadd UserObject-1-ContactObjects "ContactObject-2"
(integer) 1

So now we have:

1. Three objects stored as hashes (ContactObject-1 - ContactObject-3).
2. A set of the ids/keys of those objects (ContactObjects).
3. A counter for assigning new keys to ContactObject instances (ContactObjectsMaxOID).
4. A set of the ids/keys of those ContactObjects beloning to the user UserObject-1.

Now lets look at basic object CRUD operations via these structures.

Create

redis 127.0.0.1:6379> incr ContactObjectsMaxOID
(integer) 4
redis 127.0.0.1:6379> hmset ContactObject-4 OID 4 firstname "Jill" lastname "Doe" email "jill@nowhere.com" notes "Yet another person I know"
OK
sadd ContactObjects "ContactObject-4"

Read

redis 127.0.0.1:6379> hgetall ContactObject-4
 1) "OID"
 2) "4"
 3) "firstname"
 4) "Jill"
 5) "lastname"
 6) "Doe"
 7) "email"
 8) "jill@nowhere.com"
 9) "notes"
10) "Yet another person I know"

Update

hmset ContactObject-4 OID 4 firstname "Jill" lastname "Doe" email "jill@nowhere.com" notes "This is a new note"
OK
redis 127.0.0.1:6379> hgetall ContactObject-4
 1) "OID"
 2) "4"
 3) "firstname"
 4) "Jill"
 5) "lastname"
 6) "Doe"
 7) "email"
 8) "jill@nowhere.com"
 9) "notes"
10) "This is a new note"

Delete

redis 127.0.0.1:6379> del ContactObject-4
(integer) 1
redis 127.0.0.1:6379> hgetall ContactObject-4
(empty list or set)
redis 127.0.0.1:6379> srem ContactObjects ContactObject-4
(integer) 1
redis 127.0.0.1:6379> smembers ContactObjects
1) "ContactObject-3"
2) "ContactObject-1"
3) "ContactObject-2"

Some real-world examples

How many contact objects do we have?

redis 127.0.0.1:6379> scard ContactObjects
(integer) 3

Get all of the contacts?

Use sort to sort by OID and limit to 2 records at a time, then just hgetall on the contact OIDs returned:

redis 127.0.0.1:6379> sort ContactObjects by *->OID limit 0 2
1) "ContactObject-1"
2) "ContactObject-2"
redis 127.0.0.1:6379> sort ContactObjects by *->OID limit 2 2
1) "ContactObject-3"

Get all of the contacts for a given user?

127.0.0.1:6379> smembers UserObject-1-ContactObjects
1) "ContactObject-1"
2) "ContactObject-2"
127.0.0.1:6379> hgetall ContactObject-1
 1) "OID"
 2) "1"
 3) "firstname"
 4) "John"
 5) "lastname"
 6) "Doe"
 7) "email"
 8) "john@nowhere.com"
 9) "notes"
10) "Some person I know"
127.0.0.1:6379> hgetall ContactObject-2
 1) "OID"
 2) "2"
 3) "firstname"
 4) "Jack"
 5) "lastname"
 6) "Doe"
 7) "email"
 8) "jack@nowhere.com"
 9) "notes"
10) "Another person I know"
127.0.0.1:6379>

What about the getting the user who owns a contact?

The best thing to do here is store the user OID in the contact hash:

127.0.0.1:6379> hmset ContactObject-1 OID 1 firstname "John" lastname "Doe" email "john@nowhere.com" notes "Some person I know" UserOID 1
OK
127.0.0.1:6379> hmset ContactObject-2 OID 2 firstname "Jack" lastname "Doe" email "jack@nowhere.com" notes "Another person I know" UserOID 1
OK

Then all you need to do is fetch the key "UserObject-"+UserOID to get the contact owner.

Get all contacts where firstname is "Jock"?

Unfortunely we can't do this, as in Redis we can only query by keys and not by values, which is where the hash field firstname is stored. It is at this point I am somewhat stuck for now, and further research has yielded no path forward.

Conclusion

A big problem for me is the inability to query a hash by field value: this will make implementing the AlphaDAOProviderInterface::loadByAttribute() method impossible, and frankly that might be enough to abandon implementing AlphaDAOProviderInterface for Redis for now, and just implement AlphaCacheProviderInterface which is far more simple.