RFC: Interpolated string function calls

[rofrankel]

Summary

Allow calling functions with interpolated strings without parentheses, enabling DSL patterns like structured logging where the template, interpolation values, and optional context are passed to the function.

log:Info `Hello {name}`, {userId = 12345}
-- Desugars to: log:Info("Hello Alice", "Hello {name}", {name = "Alice"}, {userId = 12345})

Motivation

The string interpolation RFC noted the restriction on parentheses-free calls was "likely temporary while we work through string interpolation DSLs." This RFC proposes lifting that restriction with semantics designed for structured logging and similar use cases.

Key design decisions

• Parentheses-free interpolated string calls desugar to 3-4 arguments (formatted string, template, interpolation table, optional context table)
• Member expressions use nested tables for ergonomic access
• Function/method calls in templates are restricted (with a documented future extension path)
• Other expressions are allowed, with all referenced identifiers extracted

Related

• API-1080 - Structured logging APIs proposal

[rofrankel]

Edit: This pattern is now described in this RFC (not as a core part of the RFC, just as an illustration of how it might be used).

---

For completeness, one alternative proposal that came up (I can flesh this out into a separate RFC if helpful):

We could create a new Message type that wraps a string interpolation template (as a " string, presumably) and a context object, something like:

type Message = {
	template: string,
	context: { [string]: any },
	-- Renders `template` using `context`
	toString: (self: Message) -> string
}

Primary functional differences from this RFC include:

• Full encapsulation - template rendering would use only the context table, not local variables.
  • This avoids the challenges/potential limitations around method/function calls inside interpolated strings.
• Would be much easier to make backwards-compatible with print/warn/error.

Note: there are things I like about this approach, but I can't take credit for the idea - it was suggested by a colleague during a brainstorming session.

[Bottersnike]

The arguments passed to the function "called" in this way seem unintuitive, and also very specific. For this to be used in a DSL-y way (or even just by a fancy logger) it would also require that function to itself handle parsing of the { ... } blocks within the passed arguments.

Would an approach like JavaScripts template literal tag functions make more sense? When you call

log_info`Hello {name}!`

in JavaScript, it would pass the log_info function the arguments {"Hello ", "!"} followed by all the substitution values (name's value in this case) as the rest of the arguments. This means consumers of the syntax don't need to themselves worry about syntax parsing, and can just focus on either custom concatenation or complete mutation of what they were given.

In a luau-y way this might be better translated as taking two arguments, the first being a table of the string chunks (N values, N>=1) and the second being a table of the values to interpolate (N-1 values).

---

Edit: I just noticed this JS way of doing things was mentioned as a rejected alternative. The rejection reasons don't make sense to me.

Not providing the original template string is good and honestly necessary if you want any hope of this API being possible without implementing an entire luau parser, for the reasons outlined below.

A table mapping for names to values could be easily added if really desired, though I can't think of many instances outside of debuggers where it ends up necessary to know the names (as opposed to confusing, when the same template is being used but provided different locals for values).

The third rejection reason makes no sense at all; JS's "arrays" are the same as luau "tables". If multiple-arguments is the actual concern, and just poorly worded, I already addressed that here.

---

As an example of where this makes way more sense, the RFC suggests:

log:Info `{user.name} is {user.age} years old`
-- Interpolation table is {user = {name = "Alice", age = 30}}

This approach is more ergonomic for consuming code, which can use natural table access like context.user.name, and is more idiomatic to Lua/Luau's table-centric design.

To me this is absolutely not what you'd want, because now log:Info has to now not only parse out the { ... } syntax from the format string, but also handle the complex expressions that could be within it. This becomes even more of a problem with the example of:

log:Info `Sum is {a + b}`
-- Desugars to: log:Info("Sum is 30", "Sum is {a + b}", {a = 10, b = 20})

which suggests any consumer of this API would need to implement an entire luau parser to have any hope of reconstructing the desired string output!

This conundrum also then causes seemingly nonsensical restrictions being applied to the API such as just below where we see the ??? (can't use "increment()" as key twice with different values) error being thrown because we can now no longer support any mutations within the template string. If we were to consider the DSL use-case, this then causes

print(dsl`Something { dsl`Something else` }`)  

to be illegal, but one might expect many DSLs would require (or at least be made substantially nicer with) nesting.

---

Just so this comment isn't all doom and gloom, I think one approach that could be more ergonomic nice is a string.getcomponents method that returns the arguments that would be passed to a luau-style JS tag function method as described above (with normal strings just returning {"the string"}, {}); maybe even returning the original template string if you really want it for some reason, though as explained above you probably don't. While this would be a nice way to expose stuff it would cause strings to carry pretty heavy baggage when being passed around in the VM under the hood, for a feature likely to be an incredibly code path, so I'm not sure if it would actually make sense to do.

With that,

log "hello world"

and

log `hello world`

would function identically unless the body of log wanted to call string.getcomponents on its passed argument.

[hgoldstein]

The main sticking points here are:

1. The ambiguity noted by the extra parameter(s);
2. The restrictions on what can be passed to interpolated strings.

[hgoldstein]

As is, the way string interpolation appears to work is that we de-sugar:

print(`Hello {name}, it is {currtime}!`)

... to something like ...

print(string.format("Hello %*, it is %*!", name, currtime))

I'm imagining semantics like:

template `Hello {name}, it is {currtime}`

... desugaring to something like ...

-- Could also be an array for the second argument
template("Hello %*, it is %*!", name, currtime)

Would that be enough for your use case? I think it also makes the "mistaken" case of ...

print `Hello, my name is {name}`

... a little less unfortunate.

[Bottersnike]

Personally me one issue with this is you still end up needing to write your own parser to consume this API. It's not too bad, but you can't just do a simple find and replace because you need to account for %-escaping in the passed format string. I think you could cook up a nasty gsub using "([^%%])%%%*", "%1" .. replace with an extra check for if the %* is at the start of the string, but this feels pretty ugly to me.

[hgoldstein]

Yeah there's a couple decent outcomes here. My immediate thought, well before I read the RFC, is you'd get the "decomposed" interpolation, so an array of alternating string and value parts, e.g.:

log `Foo {name} bar {1 + 2}`

... becomes something like ...

log({"Foo ", name, " bar ", 3 })

I'll admit wanting to preserve the exact string came out of one of the goals of logging. Another option is that we can probably embed the actual text of the interpolated string, e.g. you'll get:

log `Foo {name} bar {1 + 2}`

... becoming ...

log("Foo {name} bar {1 + 2}", "Foo ", name, " bar ", 3 })

.... you can still reconstruct it by table.concating everything but the first argument.

[rofrankel]

From live discussion with Hunter, perhaps something like:

local a = 1
local function double(x: number)
  return x * 2
end

-- Desugars to `log:Info("The double of %* is %*", {1, 2}, {"a", "double(a)"})`
log:Info "The double of {a} is {double(a)}"`

[rofrankel]

I've updated the draft, PTAL!

[Kampfkarren]

In response to recent suggestions, I definitely don't want to have to parse out format strings (%*). One interesting thing that brings up though is that if we ever let you provide format specifiers, e.g. {name} has {money, .2f} dollars, that it becomes much easier to combine with this system if you get it back as a string passed to a function, like f(name, " has ", string.format("%.2f", money), " dollars") than it is if you get it back as the format string it would create.

[rofrankel]

I've updated this RFC with a substantially rewritten and simplified approach addressing feedback from @MagmaBurnsV, @Cooldude2606, @hgoldstein, and @Bottersnike. Thanks to @hgoldstein for the live brainstorming.

I am much happier with this version than the original version I wrote up (which came out of a 30 minute group brainstorming session that perhaps needed to be longer :) ).

The latest draft doesn't directly address @Kampfkarren's comment about potential future support for format specifiers, but perhaps it does so implicitly? E.g. if the expressions argument looked like {"name", "money, .2f"} would that work or is there some subtle problem?

[Bottersnike]

One alternative not mentioned is that you can get pretty close (albeit not with a nice interpolated string) to this behaviour just using existing functionality:

local function Log(fmt: string)
  local function log_handler(args: { any })
    print("Log event. Format string:", fmt, "Evaluated:", string.format(fmt :: any, table.unpack(args)))
  end
  return log_handler
end

local user = "Bottersnike"
Log "Hello, %*" { user }

I like the changes, though %* would make more sense than %s given the types of the arguments aren't strictly limited to strings.

---

Functions designed for parentheses-free interpolated string calls would need to be written (or updated) to accept the three-argument format.

While not an issue directly with the RFC or it's wording, it's worth noting that updating a function isn't possible in many cases. For example, print has no way of knowing if it was called with an interpolated string and should itself perform the formatting, or the user in fact called it as print(...) and passed those three values explicitly. I think for the most part any variadic function can't be updated, but any function that currently takes a single string argument could be augmented to support the additional arguments optionally.

[rofrankel]

One alternative not mentioned is that you can get pretty close (albeit not with a nice interpolated string) to this behaviour just using existing functionality:

local function Log(fmt: string)
  local function log_handler(args: { any })
    print("Log event. Format string:", fmt, "Evaluated:", string.format(fmt :: any, table.unpack(args)))
  end
  return log_handler
end

local user = "Bottersnike"
Log "Hello, %*" { user }

I like the changes, though %* would make more sense than %s given the types of the arguments aren't strictly limited to strings.

Good call, updated.

Functions designed for parentheses-free interpolated string calls would need to be written (or updated) to accept the three-argument format.

While not an issue directly with the RFC or it's wording, it's worth noting that updating a function isn't possible in many cases. For example, print has no way of knowing if it was called with an interpolated string and should itself perform the formatting, or the user in fact called it as print(...) and passed those three values explicitly. I think for the most part any variadic function can't be updated, but any function that currently takes a single string argument could be augmented to support the additional arguments optionally.

Yes this is a valid point - that's why I describe the Message approach as an alternative. I've rephased slightly to try and make that clearer.

[hgoldstein]

I think we discussed it offline, but wouldn't it be enough to (effectively) reconstruct the format string by doing something like:

local function mylog(templateparts, values)
    local fmt = table.concat(templateparts, "%*")
    print(string.format(fmt :: any, table.unpack(values)))
end

... including metadata about the values in the template aside.

[rofrankel]

This kind of works, but with two drawbacks:

1. There's collision risk for simple templates that have the same overall structure, e.g. {foo}: {bar} and {baz}: {qux}. False positives for aggregation/deduplication may in some cases be even worse than false negatives.
2. Replacing the placeholder expressions with just "%*" may hurt readability...in some cases it may be obvious what the placeholders were, but in other cases, less so.

Speaking purely as the user, these drawbacks seem significant enough that I'd rather have the placeholder expressions as well. And users who are happy without the expressions can just ignore them. I know there may be some performance hit, but as we discussed, all the identified realistic use cases for this functionality are likely going to do something much more expensive anyway with the result (e.g. make a network request), so the language performance cost is not a primary concern.

[Cooldude2606]

This new draft addresses the issues I had, although a new one has arisen.

A clarification / brief note on how literals and whitespare are past into the third argument should be included:

log `{42}`
-- this one is trivial
-- log("%*", { 42 }, { "42" })

log `{"Alice"}`
-- this one should note how escapes are handled
-- log("%*", { "Alice" }, { "\"Alice\"" })

log `{'Alice'}`
-- are single quotes maintained or converted to double quotes?
-- log("%*", { "Alice" }, { "\'Alice\'" })
-- log("%*", { "Alice" }, { "\"Alice\"" })

log `{ { ["foo"] = "bar", } }`
-- are spaces, quotes and commas maintained or trimed?
-- log("%*", { { "foo" } }, { "{foo=\"bar\"}" })
-- log("%*", { { "foo" } }, { " { [\"foo\"] = \"bar\", } " })

log `{
    user . name
}`
-- are spaces and new lines maintained or trimed?
-- log("%*", { "Alice" }, { "user.name" }
-- log("%*", { "Alice" }, { "\n\tuser . name\n" }

[rofrankel]

This new draft addresses the issues I had, although a new one has arisen.

A clarification / brief note on how literals and whitespare are past into the third argument should be included:

log `{42}`
-- this one is trivial
-- log("%*", { 42 }, { "42" })

log `{"Alice"}`
-- this one should note how escapes are handled
-- log("%*", { "Alice" }, { "\"Alice\"" })

log `{'Alice'}`
-- are single quotes maintained or converted to double quotes?
-- log("%*", { "Alice" }, { "\'Alice\'" })
-- log("%*", { "Alice" }, { "\"Alice\"" })

log `{ { ["foo"] = "bar", } }`
-- are spaces, quotes and commas maintained or trimed?
-- log("%*", { { "foo" } }, { "{foo=\"bar\"}" })
-- log("%*", { { "foo" } }, { " { [\"foo\"] = \"bar\", } " })

log `{
    user . name
}`
-- are spaces and new lines maintained or trimed?
-- log("%*", { "Alice" }, { "user.name" }
-- log("%*", { "Alice" }, { "\n\tuser . name\n" }

Thanks, good point, updated.

[hgoldstein]

... use a logging library that implements its own template parsing at runtime (duplicating language functionality).

I'm not sure that duplicating language functionality is the issue here: it's more that this is also error prone in its own way. For example:

log:Info("Hello, my name is: {{nome}}", { name = "Hunter" })

... whereas we already have a mechanism for constructing strings that can provide analysis input (intellisense, error checking): it's more missed opportunity than duplicating language features being a problem.

[andyfriesen]

Would it be better (or worse?) if the desugaring instead included byte offsets into the original string for each substitution?

log`{timestamp}: Count is {count}.  Total is {count + total}`
-- desugars to
log("{timestamp}: Count is {count}.  Total is {count + total}", {timestamp, count, count + total},
    -- (offset, length) pairs for each substitution
    {{0, 11}, {22, 7}, {45, 15}}
)

This trivializes any custom parsing you might want to do.

[andyfriesen]

One alternative not mentioned is that you can get pretty close (albeit not with a nice interpolated string) to this behaviour just using existing functionality:

local function Log(fmt: string)
  local function log_handler(args: { any })
    print("Log event. Format string:", fmt, "Evaluated:", string.format(fmt :: any, table.unpack(args)))
  end
  return log_handler
end

local user = "Bottersnike"
Log "Hello, %*" { user }

I like this interface quite a lot.

I'd like to see this approach added to the "Alternatives" section alongside a strong argument for why it is unviable.