Redis

(notes from Redis in Action, the book is pretty old and does not cover some latest updates like redis 3.0 comes with cluster support)

Data Types

five different data structure types:

• STRING: Strings, integers, or floatingpoint values. Operate on the whole string, parts, increment/decrement the integers and floats.
• LIST: Linked list of strings. Push or pop items from both ends, trim based on offsets, read individual or multiple items, find or remove items by value.
• SET: Unordered collection of unique strings. Add, fetch, or remove individual items, check membership, intersect, union, difference, fetch random items.
• HASH: Unordered hash table of keys to values. Add, fetch, or remove individual items, fetch the whole hash.
• ZSET (sorted set): Ordered mapping of string members to floating-point scores, ordered by score. Add, fetch, or remove individual values, fetch items based on score ranges or member value.

Commands for Data Types

STRING

• GET: Fetches the data stored at the given key
• SET: Sets the value stored at the given key
• DEL: Deletes the value stored at the given key (works for all types)
• INCR: INCR key-name—Increments the value stored at the key by 1
• DECR: DECR key-name—Decrements the value stored at the key by 1
• INCRBY: INCRBY key-name amount—Increments the value stored at the key by the provided integer value
• DECRBY: DECRBY key-name amount—Decrements the value stored at the key by the provided integer value
• INCRBYFLOAT: INCRBYFLOAT key-name amount—Increments the value stored at the key by the provided float value
• APPEND: APPEND key-name value—Concatenates the provided value to the string already stored at the given key
• GETRANGE: GETRANGE key-name start end—Fetches the substring, including all characters from the start offset to the end offset, inclusive
• SETRANGE: SETRANGE key-name offset value—Sets the substring starting at the provided offset to the given value
• GETBIT: GETBIT key-name offset—Treats the byte string as a bit string, and returns the value of the bit in the string at the provided bit offset
• SETBIT: SETBIT key-name offset value—Treats the byte string as a bit string, and sets the value of the bit in the string at the provided bit offset
• BITCOUNT: BITCOUNT key-name [start end]—Counts the number of 1 bits in the string, optionally starting and finishing at the provided byte offsets
• BITOP: BITOP operation dest-key key-name [key-name ...]—Performs one of the bitwise operations, AND, OR, XOR, or NOT, on the strings provided, storing the result in the destination key

> set hello world
> get hello
"hello"
> del hello
> get hello
(nil)

LIST

• RPUSH: RPUSH key-name value [value ...]—Pushes the value(s) onto the right end of the list
• LPUSH: LPUSH key-name value [value ...]—Pushes the value(s) onto the left end of the list
• RPOP: RPOP key-name—Removes and returns the rightmost item from the list
• LPOP: LPOP key-name—Removes and returns the leftmost item from the list
• LINDEX: LINDEX key-name offset—Returns the item at the given offset
• LRANGE: LRANGE key-name start end—Returns the items in the list at the offsets from start to end, inclusive
• LTRIM: LTRIM key-name start end—Trims the list to only include items at indices between start and end, inclusive
• BLPOP: BLPOP key-name [key-name ...] timeout—Pops the leftmost item from the first non-empty LIST, or waits the timeout in seconds for an item
• BRPOP: BRPOP key-name [key-name ...] timeout—Pops the rightmost item from the first non-empty LIST, or waits the timeout in seconds for an item
• RPOPLPUSH: RPOPLPUSH source-key dest-key—Pops the rightmost item from the source and LPUSHes the item to the destination, also returning the item to the user
• BRPOPLPUSH: BRPOPLPUSH source-key dest-key timeout—Pops the rightmost item from the source and LPUSHes the item to the destination, also returning the item to the user, and waiting up to the timeout if the source is empty

> rpush list-key item
> rpush list-key item2
> rpush list-key item
> lrange list-key 0 -1
1) "item"
2) "item2"
3) "item"
> lindex list-key 1
"item2"
> lpop list-key
"item"
> lrange list-key 0 -1
1) "item2"
2) "item"

SET

• SADD: SADD key-name item [item ...]—Adds the items to the set and returns the number of items added that weren’t already present
• SREM: SREM key-name item [item ...]—Removes the items and returns the number of items that were removed
• SISMEMBER: SISMEMBER key-name item—Returns whether the item is in the SET
• SCARD: SCARD key-name—Returns the number of items in the SET
• SMEMBERS: SMEMBERS key-name—Returns all of the items in the SET as a Python set
• SRANDMEMBER: SRANDMEMBER key-name [count]—Returns one or more random items from the SET. When count is positive, Redis will return count distinct randomly chosen items, and when count is negative, Redis will return count randomly chosen items that may not be distinct.
• SPOP: SPOP key-name—Removes and returns a random item from the SET
• SMOVE: SMOVE source-key dest-key item—If the item is in the source, removes the item from the source and adds it to the destination, returning if the item was moved

combine multiple SETs:

• SDIFF: SDIFF key-name [key-name ...]—Returns the items in the first SET that weren’t in any of the other SETs (mathematical set difference operation)
• SDIFFSTORE: SDIFFSTORE dest-key key-name [key-name ...]—Stores at the dest-key the items in the first SET that weren’t in any of the other SETs (mathematical set difference operation)
• SINTER: SINTER key-name [key-name ...]—Returns the items that are in all of the SETs (mathematical set intersection operation)
• SINTERSTORE: SINTERSTORE dest-key key-name [key-name ...]—Stores at the dest-key the items that are in all of the SETs (mathematical set intersection operation)
• SUNION: SUNION key-name [key-name ...]—Returns the items that are in at least one of the SETs (mathematical set union operation)
• SUNIONSTORE: SUNIONSTORE dest-key key-name [key-name ...]—Stores at the dest-key the items that are in at least one of the SETs (mathematical set union operation)

> sadd set-key item
(integer) 1

> sadd set-key item2
> sadd set-key item3
> sadd set-key item
(integer) 0
> smembers set-key
1) "item"
2) "item2"
3) "item3"
> sismember set-key item4
(integer) 0
> sismember set-key item
(integer) 1
> srem set-key item2
(integer) 1
> srem set-key item2
(integer) 0
> smembers set-key
1) "item"
2) "item3"

HASH

• HSET: Stores the value at the key in the hash
• HGET: Fetches the value at the given hash key
• HGETALL: Fetches the entire hash
• HDEL: Removes a key from the hash, if it exists
• HMGET: HMGET key-name key [key ...]—Fetches the values at the fields in the HASH
• HMSET: HMSET key-name key value [key value ...]—Sets the values of the fields in the HASH
• HDEL: HDEL key-name key [key ...]—Deletes the key-value pairs in the HASH, returning the number of pairs that were found and deleted
• HLEN: HLEN key-name—Returns the number of key-value pairs in the HASH
• HEXISTS: HEXISTS key-name key—Returns whether the given key exists in the HASH
• HKEYS: HKEYS key-name—Fetches the keys in the HASH
• HVALS: HVALS key-name—Fetches the values in the HASH
• HGETALL: HGETALL key-name—Fetches all key-value pairs from the HASH
• HINCRBY: HINCRBY key-name key increment—Increments the value stored at the given key by the integer increment
• HINCRBYFLOAT: HINCRBYFLOAT key-name key increment—Increments the value stored at the given key by the float increment

> hset hash-key sub-key1 value1
(integer) 1
> hset hash-key sub-key2 value2
(integer) 1
> hset hash-key sub-key1 value1
(integer) 0
> hgetall hash-key
1) "sub-key1"
2) "value1"
3) "sub-key2"
4) "value2"
> hdel hash-key sub-key2
(integer) 1
> hdel hash-key sub-key2
(integer) 0
> hget hash-key sub-key1
"value1"
> hgetall hash-key
1) "sub-key1"
2) "value1"

ZSET

• ZADD: ZADD key-name score member [score member ...]—Adds members with the given scores to the ZSET
• ZREM: ZREM key-name member [member ...]—Removes the members from the ZSET, returning the number of members that were removed
• ZCARD: ZCARD key-name—Returns the number of members in the ZSET
• ZINCRBY: ZINCRBY key-name increment member—Increments the member in the ZSET
• ZCOUNT: ZCOUNT key-name min max—Returns the number of members with scores between the provided minimum and maximum
• ZRANK: ZRANK key-name member—Returns the position of the given member in the ZSET
• ZSCORE: ZSCORE key-name member—Returns the score of the member in the ZSET
• ZRANGE: ZRANGE key-name start stop [WITHSCORES]—Returns the members and optionally the scores for the members with ranks between start and stop
• ZREVRANK: ZREVRANK key-name member—Returns the position of the member in the ZSET, with members ordered in reverse
• ZREVRANGE: ZREVRANGE key-name start stop [WITHSCORES]—Fetches the given members from the ZSET by rank, with members in reverse order
• ZRANGEBYSCORE: ZRANGEBYSCORE key min max [WITHSCORES] [LIMIT offset count]—Fetches the members between min and max
• ZREVRANGEBYSCORE: ZREVRANGEBYSCORE key max min [WITHSCORES] [LIMIT offset count]—Fetches the members in reverse order between min and max
• ZREMRANGEBYRANK: ZREMRANGEBYRANK key-name start stop—Removes the items from the ZSET with ranks between start and stop
• ZREMRANGEBYSCORE: ZREMRANGEBYSCORE key-name min max—Removes the items from the ZSET with scores between min and max
• ZINTERSTORE: ZINTERSTORE dest-key key-count key [key ...] [WEIGHTS weight [weight ...]] [AGGREGATE SUM|MIN|MAX]—Performs a SET-like intersection of the provided ZSETs
• ZUNIONSTORE: ZUNIONSTORE dest-key key-count key [key ...] [WEIGHTS weight [weight ...]] [AGGREGATE SUM|MIN|MAX]—Performs a SET-like union of the provided ZSETs

> zadd zset-key 728 member1
(integer) 1
> zadd zset-key 982 member0
(integer) 1
> zadd zset-key 982 member0
(integer) 0
> zrange zset-key 0 -1 withscores
1) "member1"
2) "728"
3) "member0"
4) "982"
> zrangebyscore zset-key 0 800 withscores
1) "member1"
2) "728"
> zrem zset-key member1
(integer) 1
> zrem zset-key member1
(integer) 0
> zrange zset-key 0 -1 withscores
1) "member0"
2) "982"

Publish/Subscribe

• SUBSCRIBE: SUBSCRIBE channel [channel ...]—Subscribes to the given channels
• UNSUBSCRIBE: UNSUBSCRIBE [channel [channel ...]]—Unsubscribes from the provided channels, or unsubscribes all channels if no channel is given
• PUBLISH: PUBLISH channel message—Publishes a message to the given channel
• PSUBSCRIBE: PSUBSCRIBE pattern [pattern ...]—Subscribes to messages broadcast to channels that match the given pattern
• PUNSUBSCRIBE: PUNSUBSCRIBE [pattern [pattern ...]]—Unsubscribes from the provided patterns, or unsubscribes from all subscribed patterns if none are given

Sort

• SORT: SORT source-key [BY pattern] [LIMIT offset count] [GET pattern [GET pattern ...]] [ASC|DESC] [ALPHA] [STORE dest-key]—Sorts the input LIST, SET, or ZSET according to the options provided, and returns or stores the result

Expiring keys

• PERSIST: PERSIST key-name—Removes the expiration from a key
• TTL: TTL key-name—Returns the amount of time remaining before a key will expire
• EXPIRE: EXPIRE key-name seconds—Sets the key to expire in the given number of seconds
• EXPIREAT: EXPIREAT key-name timestamp—Sets the expiration time as the given Unix timestamp
• PTTL: PTTL key-name—Returns the number of milliseconds before the key will expire (available in Redis 2.6 and later)
• PEXPIRE: PEXPIRE key-name milliseconds—Sets the key to expire in the given number of milliseconds (available in Redis 2.6 and later)
• PEXPIREAT: PEXPIREAT key-name timestamp-milliseconds—Sets the expiration time to be the given Unix timestamp specified in milliseconds (available in Redis 2.6 and later)

Persistence

there are two different ways of persisting data to disk. One is a method called snapshotting that takes the data as it exists at one moment in time and writes it to disk. The other method is called AOF, or append-only file, and it works by copying incoming write commands to disk as they happen. These methods can be used together, separately, or not at all in some circumstances.

snapshotting options:

save 60 1000
stop-writes-on-bgsave-error no
rdbcompression yes
dbfilename dump.rdb

append-only options:

appendonly no
appendfsync everysec
no-appendfsync-on-rewrite no
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb

shared options:

dir ./

Replication

What happens when a slave connects to a master:

Step	Master operations	Slave operations
1	(waiting for a command)	(Re-)connects to the master; issues the SYNC command
2	Starts BGSAVE operation; keeps a backlog of all write commands sent after BGSAVE	Serves old data (if any), or returns errors to commands (depending on configuration)
3	Finishes BGSAVE; starts sending the snapshot to the slave; continues holding a backlog of write commands	Discards all old data (if any); starts loading the dump as it’s received
4	Finishes sending the snapshot to the slave; starts sending the write command backlog to the slave	Finishes parsing the dump; starts responding to commands normally again
5	Finishes sending the backlog; starts live streaming of write commands as they happen	Finishes executing backlog of write commands from the master; continues executing commands as they happen

Handling system failures

$ redis-check-aof
Usage: redis-check-aof [--fix] <file.aof>
$ redis-check-dump
Usage: redis-check-dump <dump.rdb>

Redis transactions

We begin our transaction by calling the special command MULTI, passing our series of commands, followed by EXEC.

In the process of listing, we’ll use a Redis operation called WATCH, which we combine with MULTI and EXEC, and sometimes UNWATCH or DISCARD

Redis doesn’t lock data during WATCH. Instead, Redis will notify clients if someone else modified the data first, which is called optimistic locking