Skip to content

Fix!: Inconsistent behaviour when a macro returns a list containing a single array vs multiple arrays#5037

Merged
erindru merged 1 commit intomainfrom
erin/macro-each-reduce
Jul 28, 2025
Merged

Fix!: Inconsistent behaviour when a macro returns a list containing a single array vs multiple arrays#5037
erindru merged 1 commit intomainfrom
erin/macro-each-reduce

Conversation

@erindru
Copy link
Collaborator

@erindru erindru commented Jul 28, 2025

Closes #4536

Prior to this, the following behavior occurred:

>>> from sqlmesh.core.macros import MacroEvaluator
>>> from sqlglot import parse_one
>>> me = MacroEvaluator()

>>> me.transform(parse_one("select @each([1], a -> [@a])")).sql()
'SELECT ARRAY(1)' 

>>> me.transform(parse_one("select @reduce([[1]], (x,y) -> x + y)")).sql()
'SELECT ARRAY(1)' 

>>> me.transform(parse_one("select @reduce(@each([1, 2], a -> [@a]), (x,y) -> x + y)")).sql() 
'SELECT ARRAY(1) + ARRAY(2)'

>>> me.transform(parse_one("select @reduce(@each([1], a -> [@a]), (x,y) -> x + y)")).sql() 
'SELECT 1'

That last item is unexpected, it should be SELECT ARRAY(1) because @each returns 1 item of type array.

Unfortunately, _norm_var_arg_lambda() assumes this item is varargs and unpacks it, rather than passing it as-is.

Sometimes, we do actually want it to be unpacked though. If we didnt, select @each([1, 2], a -> [@a]) would result in SELECT ARRAY(ARRAY(1), ARRAY(2)) instead of SELECT ARRAY(1), ARRAY(2).

So this PR wraps the output of a macro function in an extra exp.Array to prevent _norm_var_arg_lambda() from treating it as varargs, but only if it's being used as input to another function, which preserves the semantics of still unpacking it for a SELECT column list.

@tobymao
Copy link
Contributor

tobymao commented Jul 28, 2025

is this breaking?

@erindru erindru changed the title Fix: Inconsistent behaviour when a macro returns a list containing a single array vs multiple arrays Fix!: Inconsistent behaviour when a macro returns a list containing a single array vs multiple arrays Jul 28, 2025
@erindru
Copy link
Collaborator Author

erindru commented Jul 28, 2025

I guess every bug fix is breaking if someone was depending on the buggy behaviour

@erindru erindru merged commit 04d199b into main Jul 28, 2025
27 checks passed
@erindru erindru deleted the erin/macro-each-reduce branch July 28, 2025 21:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

each passed to @reduce has inconsistent behaviour

3 participants