redis-lua unit testing

[benjaminweb]

You've done a little but amazing piece of code with redis-lua.
It does a pretty job for a dozen of redis lua scripts for me.

I am glueing together some kind of unit testing for those at the moment.

It works basically. However, the syntax of redis-lua causes hiccup
with %arg and %return syntax.
Can we wrap it into Lua --[[ and --]] comment?

The relevant code seems to be this here:

regions.py:433 r'^\s*%key\s+(?P<name>[\w\d_]+)\s*$',
regions.py:453
r'^\s*%arg\s+(?P<name>[\w\d_]+)(\s+(?P<type>[\w\d_]+))?\s*$',

I considered implementing unit tests for Lua in Python directly by means
of lupa and testing.redis.
However, I feel this introduces unnecessary complexity.

For now, I think the simplest path would be lua directly.

What do you think?

[catwell]

@benjaminweb Have you tried if the Lua comment syntax does not work already?

The Python code looks line by line, and I don't think it expects the lines starting with % to be at the top of the file. So if you enclose then in a Lua comment block, I suppose it might just work fine.

[benjaminweb]

I've updated the gist to incorporate the comment syntax.

lua-5.1 simple_test_web.lua raises

lua-5.1: ./inexisting_query.lua:5: variable 'redis' is not declared
stack traceback:
    [C]: in function 'error'
    /usr/local/share/lua/5.1/pl/strict.lua:84: in function </usr/local/share/lua/5.1/pl/strict.lua:67>
    ./inexisting_query.lua:5: in main chunk
    [C]: in function 'require'
    simple_test_web.lua:68: in main chunk
    [C]: ?

[catwell]

That's an issue that does not depend on the Python redis-lua module. The Lua script does not have the redis variable in its environment. A simple way to fix this is to make it global, a cleaner way is to call the Lua script with a custom environment.

Moreover, you have another issue, which is that inexisting_query.lua is not a normal Redis script anyway (it just defines a local function, which is to say it does nothing at all). The main script will not be able to call inexisting_query either (because it is local to the script), and it should not have to know that this function exists anyway.

A clean way to support unit testing of Redis script would require work inside fakeredis itself. It would not use require to load scripts, it would use load with a custom environment. But sadly I don't have the time to code that up right now (and this is far from being my priority since I basically have not been using Redis that much for the last 3 years).

[benjaminweb]

@catwell, thanks for your realistic perspective.

As far as I know, redis lua scripts may only use local functions. => no direct import from other Lua script (this must be to deny any imports into redis lua scripts itself?).

I've updated inexisting_query.lua again: Now it calls the local function inexisting_query.

Would it make sense to implement the custom environment in Python with testing.redis and Lupa?

Where do you guess the major chunk of effort to be, @catwell?

@ereOn what do you think the unit testing should happen? In Python or in Lua directly?
A concisely defined interface is everything.

[catwell]

As far as I know, redis lua scripts may only use local functions.

Yes, but they have redis in scope by default.

Would it make sense to implement the custom environment in Python

When I was talking about a custom Lua environment (see getenv / setenv in 5.1, _ENV in later versions) to load the chunk in.

I've updated inexisting_query.lua again: Now it calls the local function inexisting_query.

It calls the function in a way that can only work with the Python redis-lua module. You wrote:

inexisting_query(query, number)

but neither query or number exist in the Redis script from the point of view of Lua. The Python module probably transforms the script before sending it to Redis somehow.

I am wondering if it is possible to get the transformed script from the Python module. Something without the % lines and with ARGV / KEYS. If I implement something in fakeredis, it will be this (the native format of Redis scripts).

[ereOn]

@benjaminweb I've been doing all my "unit" testing (not sure I can really call it that) from the Python side.

I basically define my root LUA scripts as functions (each contains one function) and I make real calls to Redis and ensure the results are as I expect them. To be honest this is more like integration testing but in my setup, it got me to catch very nasty regressions so I'm pretty happy with that approach. I would not mind having a real unit testing approach available, but I'm afraid my LUA-ecosystem knowledge is too limited to even know where to begin.

@catwell It is possible to get the generated script content indeed : look at the Script.render() method. This is what the library uses to load the script into Redis too.

[benjaminweb]

@ereOn I've done it in the same manner like you outlined.
Suppose a more complex Lua script > 150 LOC that is the core business logic.
How would you feel about having only integration tests in place?
Yes, currently the integration tests are designed to enable me kind of tracing the error down to a specific part of the Lua script. However, it's not testing the smallest thing that can go wrong: each subfunction itself.

So, @ereOn what if we render that script and extract the functions itself, hand over them to Lupa and testing.redis?
Which way could we get most of the result (≠ all) but get rid (typo: red ;) ) of majority of effort?

[ereOn]

@benjaminweb You don't have to convince me on the merits of unit testing : I'm pretty much a fan myself too, be assured ;)

It is more out of technical difficulties that I did not go down that path yet for my LUA scripts.

You can indeed render the script and extract the functions, but please note that you can also split your LUA files with %include statements, and just load the sub-files one by one. render() does work on all LUA scripts, not just the Python-facing ones.

[benjaminweb]

I've not been aware of the %include statement. Can I use this to atomise the big function into small and smaller ones? Try it, @benjaminweb!

Then, what would hold one back from testing the smallest possible function in python?
➔ breaking up the > 150 LOC into smaller ones.

However, how will complexity increase? Is that a net gain or loss?

This screams for some crispy examples (yup, I've seen [integration_tests]).(https://github.com/ereOn/redis-lua/blob/master/integration_tests/__init__.py)

[benjaminweb]

@ereOn, would you mind demonstrating by example how an include statement works?

https://github.com/ereOn/redis-lua/blob/master/integration_tests/__init__.py#L234-L240

If I can isolate functions into their own files, I could test those lua files directly from python.
Can I? How?

Where do you see potential for further development of redis_lua?
Could we even think of firing up the redis lua debugger?

[benjaminweb]

@catwell, you wrote

A clean way to support unit testing of Redis script would require work inside fakeredis itself. It would not use require to load scripts, it would use load with a custom environment.

What do I need to do to take this a step forward?
What do you mean by loading a custom environment? Excellent would be a minimum working example!

Any other ideas highly appreciated!

[catwell]

What do I need to do to take this a step forward?

If you know Lua well enough you can try to implement it, but it's some work. Basically the idea is that you need to implement EVAL in fakeredis.

What do you mean by loading a custom environment?

See here and here.

Here is the idea of how it should work, except it should be within fakeredis, using the data structures of fakeredis, and obviously a lot more feature-complete...

local data = {foo = "bar"}

local redis = {}
redis.call = function(cmd, ...)
    local arg = {...}
    if cmd == "get" then
        return data[arg[1]]
    else
        error("unsupported command: " .. cmd)
    end
end

local env = {redis = redis}
local function eval(script)
    return load(script, "script", "t", env)()
end

local script = [[return redis.call("get", "foo")]]
print(eval(script))

(Well, actually it's even slightly more complicated than that, because here I used Lua 5.3 syntax, but Redis uses Lua 5.1. And in Lua 5.1 environments work a bit differently (it changed in 5.2).)

[benjaminweb]

Thanks, @catwell for your elaborate answer.
I can't promise anything right now.