Custom Redis Command: Performance of C vs Lua

Dec 24, 2012

Over the last few days, we've been writing a custom Redis command in C. The command, xdiff, is tailored to a specific use-case: diff'ing a set from a sorted set with an offset and count. A reasonable question is: why not just write it in Lua?

Keeping in mind the dangers of such synthetic benchmarks, I decided to compare the C implementation to one written in Lua. The full C implementation can be seen at the end of part 2. Here's the Lua version:

local zset = KEYS[1]
local diff = KEYS[2]
local from = ARGV[1]
local to = tonumber(ARGV[2])
local count = to - from
local results = {}

while true do
local values = redis.call('zrevrange', zset, from, to)
if #values == 0 then
return results

for index,value in ipairs(values) do
if redis.call('sismember', diff, value) == 0 then
table.insert(results, value)
if #results == count then
return results
from = to + 1
to = from + count

Here's our benchmark:

require 'redis'
require 'benchmark'

r = Redis.new

script = <<-eos
#...the lua version

sha = r.script :load, script

values = Array.new(100000) {|i| "value:#{i}"}
r.zadd 'zset', Array.new(100000) {|i| [i, values[i]] }
r.sadd '1percent', values.sample(1000)
r.sadd '25percent', values.sample(25000)
r.sadd '50percent', values.sample(50000)
r.sadd '75percent', values.sample(75000)

sets = ['1percent', '25percent', '50percent', '75percent']

Benchmark.bmbm(15) do |x|
sets.each do |set|
x.report "c - #{set}" do
1000.times { r.xdiff 'zset', set, rand(50000), rand(500) + 100 }
sets.each do |set|
x.report "lua - #{set}" do
1000.times do
from = rand(50000)
to = from + rand(500) + 100
r.evalsha sha, ['zset', set], [from, to]

Essentially, I'm interested in finding out how each performs based on the number of similarities between our sorted set and our set. The more they have in common, the more work is involved (since we need to find X that exist in zset which don't exist in our set).

On my somewhat underpowered laptop, the result is:

user     system      total        real
c - 1percent 4.130000 0.460000 4.590000 ( 4.321763)
c - 25percent 3.990000 0.410000 4.400000 ( 4.910855)
c - 50percent 3.910000 0.350000 4.260000 ( 4.995748)
c - 75percent 4.020000 0.380000 4.400000 ( 5.493845)
lua - 1percent 4.000000 0.360000 4.360000 ( 6.992061)
lua - 25percent 4.010000 0.350000 4.360000 ( 7.670618)
lua - 50percent 4.080000 0.390000 4.470000 ( 9.089831)
lua - 75percent 4.050000 0.370000 4.420000 ( 12.835829)

Unsurprisingly, the C implementation is faster and is much better at dealing with a large overlap. The Lua version is still plenty fast though. The exact performance difference will depend on what you're doing. Simple queries aren't likely to gain much from being written in C. For a complex query though, we've seen more than a 2x performance improvement (but we still favor Lua in all but extreme cases).