Skip to content

RFC: Extend buffer with a metatable #155

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
vegorov-rbx wants to merge 6 commits into
base: master
Choose a base branch
from
Draft

RFC: Extend buffer with a metatable #155

Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
5a67cf1
Initial proposal
vegorov-rbx Dec 4, 2025
27a1865
Address the first batch of comments on the RFC
vegorov-rbx Dec 5, 2025
375a0a1
Provide notes on this feature use in potential separate data section …
vegorov-rbx Dec 5, 2025
8fe1ebd
Extend motivation to note how buffers already have a metatable, note …
vegorov-rbx Dec 12, 2025
81580dd
Update docs/buffer-metatable.md
vegorov-rbx Dec 16, 2025
42a5765
Use setmetatable as the proposed typing solution
vegorov-rbx Dec 16, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
149 changes: 149 additions & 0 deletions docs/buffer-metatable.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,149 @@
# Extend buffer with a metatable

## Summary

Extend `buffer.create` with a second argument which provides a metatable for that buffer.
The metatable and some of its keys will be frozen for potential improvements in access.

## Motivation

Buffer type today has a single metatable provided by the embedder and shared by all buffer instances.

This limits the flexibility as all buffers share the same behavior and those behaviors cannot be defined by user in Luau.
Even for the embedder, defining the methods in Luau is challenging and host language definitions are used instead.

Providing a buffer object with an individual metatable will allow buffer objects to have properties (direct or computed), methods and operators, specific to certain instances.

This will allow the behavior to be defined and extended on the Luau side, compared to userdata that is strictly defined by the embedder.

This extension preserves the simple small core of the Luau language while providing a flexible extension point for developers.

Such buffers can also be made to match the structure of the host data types and to handle FFI cases.

In a way, this will provide an alternative to luajit 'cdata' in Luau:

```luau
-- A float4 in a buffer
Copy link
Contributor

@alexmccord alexmccord Dec 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not a float4.

Suggested change
-- A float4 in a buffer
-- A vector4 in a buffer

local mt = {
__index = function(b, field)
if field == "x" then return buffer.readf32(b, 0)
elseif field == "y" then return buffer.readf32(b, 4)
elseif field == "z" then return buffer.readf32(b, 8)
elseif field == "w" then return buffer.readf32(b, 12)
else error("unknown field") end
end,
__newindex = function(b, field, value)
if field == "x" then buffer.writef32(b, 0, value)
elseif field == "y" then buffer.writef32(b, 4, value)
elseif field == "z" then buffer.writef32(b, 8, value)
elseif field == "w" then buffer.writef32(b, 12, value)
else error("unknown field") end
end
}

local buf = buffer.create(16, mt)

buf.x = 2
buf.y = 4

assert(buf.x + buf.y + buf.z == 6)
```

Or alternatively:
```luau
local mt = {
__index = {
x = function(b) return buffer.readf32(b, 0) end,
y = function(b) return buffer.readf32(b, 4) end,
z = function(b) return buffer.readf32(b, 8) end,
w = function(b) return buffer.readf32(b, 12) end,
setx = function(b, value) buffer.writef32(b, 0, value) end,
sety = function(b, value) buffer.writef32(b, 4, value) end,
setz = function(b, value) buffer.writef32(b, 8, value) end,
setw = function(b, value) buffer.writef32(b, 12, value) end,
magnitude = function(b) return math.sqrt(b.x * b.x + b.y * b.y + b.z * b.z + b.w * b.w) end,
normalize = function(b) return ... end,
},
}

local buf = buffer.create(16, mt)

buf:setx(2)
buf:sety(4)
buf:normalize()

local xn = buf:x()
```

Having all the flexibility of a metatable, developer is free to define whatever properties and behaviors they want and interpret their buffer data in a form suitable for them.

## Design

`buffer.create(size: number, metatable: table?): buffer`
Copy link
Contributor

@alexmccord alexmccord Dec 18, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With the new setmetatable suggestion, this would be

Suggested change
`buffer.create(size: number, metatable: table?): buffer`
`buffer.create<M>(size: number, metatable: M): setmetatable<buffer, M>`

But because currently Luau reports an error when you supply too few arguments even if the tail end of the parameters are generics that can range over nils, you have to turn it into an overloaded function type:

Suggested change
`buffer.create(size: number, metatable: table?): buffer`
`buffer.create(size: number): buffer`
`buffer.create<M>(size: number, metatable: M): setmetatable<buffer, M>`

This is still inconsistent with the current typing algebra for setmetatable though: the function setmetatable cannot be called with buffer but the type function setmetatable can? The behavior of every type functions have been modeled after their cousin function in the value namespace, making this new equation an odd duck. I really want to avoid any divergences in this aspect.

Then there's also the question of what setmetatable<buffer, nil> would even do? If we base it on how setmetatable<{}, nil> behaves (e.g. setmetatable({}, nil)), then by identity with the substitution of M=nil, this would presumably reduce to buffer. Surely we would like for you to get a type error where the type function setmetatable cannot be called on a buffer that have already been created without one, right? As it currently stands with my assumption, this is unsound since setmetatable<setmetatable<buffer, nil>, {...some metatable...}> will gladly report zero errors since the inner setmetatable just reduces to buffer without any provenance showing that setmetatable have already been called on it.

It might seem absurd to care if someone wrote that directly (and it is), but the type system can (and will) derive M=nil (or some variation thereof where M is a union with nil) regardless of the syntax itself, and type functions are distributive over every type arguments: F<A | B, C | D> = F<A, C> | F<B, C> | F<A, D> | F<B, D>. All you need to do is replace C | D with {...some metatable...} | nil, then by distributivity you get setmetatable<buffer, {...some metatable...}> | setmetatable<buffer, nil>, which reduces to { buffer, @metatable {...some metatable...} } | buffer. Part of the reason for having a type system is to detect logic errors, and setmetatable being called more than once on a buffer categorically is a logic error, whether written by user or through type inference.

If you make setmetatable<buffer, nil> not reduce to buffer to avoid being unsound, this makes it different from the first overload (namely buffer.create(size: number): buffer) which would mean the first overload must return setmetatable<buffer, nil>, not buffer, so this should really be:

Suggested change
`buffer.create(size: number, metatable: table?): buffer`
`buffer.create(size: number): setmetatable<buffer, nil>`
`buffer.create<M>(size: number, metatable: M): setmetatable<buffer, M>`

Which then comes back to my first problem I mentioned where we now have one type function that diverged behaviorally from their value cousin... This could be alleviated by allowing setmetatable(buffer.create(16), my_buffer_api) (and treat my_buffer_api as if it had a __metatable field, which prevents a second setmetatable call), which makes the typing algebra consistent with its cousin.

The third issue is how this relates to subtyping: setmetatable<buffer, M> is a subtype of buffer (not the other way around), which makes it too easy to pass any arbitrary buffer into a function that takes a buffer. This is fine, but that function could do anything to the buffer in question. Literally anything. It's the equivalent of char* in C where you could just... change one byte in the buffer and now your whole data structure is straight up garbage data. This is the single biggest concern I have with this proposal: it tries to pretend buffer are not always a opaque blob of data, but it is! This potentially introduces a vector that creates more buggy programs. This is why I am against this RFC in general, we shouldn't make it seem like the type system can give buffers a type. If we do, people will start replacing tables with buffers, and end up mixing all sorts of different buffer types. We don't even have a notion of a readonly buffer, or a standard way to do type refinements on buffers if you try to simulate Rust enums in it, whereas tables can already do this. I have always viewed buffer as ephemeral data that is usually owned by a function or some data structure that needs buffers for performance reasons only, not intended to be exposed to the public. (put another way: this reeks of stringly-typed APIs)

Copy link
Contributor

@alexmccord alexmccord Dec 18, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's also harder to make illegal states unrepresentable:

local buf_mt = {
  __index = {
    x = function(b) return buffer.readf32(b, 0) end
  },
  __newindex = {
    x = function(b, v) buffer.writestring(b, 0, v) end
  },
}

local b = buffer.create(5, buf_mt)

local x: number = b.x
b.x = "five"

This makes shape types fundamentally awkward and could even be something absurd like { read x: number, write x: string }. Right now, in this function we infer t: { x: number }, and I think everyone wants this to report an error that you cannot assign string to x.

local function f(t)
    local x: number = t.x
    t.x = "five"
end

It also breaks typestates, which currently assumes that any read after t.x = "five" is of type string (or really, the fresh type for the string literal).


Buffer construction will accept a second argument which will be a metatable for the buffer object.

This metatable will be frozen.
In addition to that, `__index` and `__newindex` fields will also be frozen if they are a table.
Table freezing will have the same limitation as `table.freeze`, throwing an error if table metatable is locked.
Any of these tables can be frozen before the call.

When `__index` or `__newindex` is a function, VM will be allowed to ignore changes to the environment of those function.
This is similar to function inlining functionality we have where an inlined function will no longer respect its environment.

The freezing is performed to provide additional guarantees for field access and other metamethod evaluation to unlock potential optimization opportunities.
This is similar to limitations placed on 'cdata' by luajit.
Having said that, this RFC doesn't make a promise of a particular implementation making those optimizations, so should be viewed as a buffer object usability improvement first.

Any chained metatables on table fields are not modified.
This provides an option to have a more dynamic behavior, but giving up on potential performance improvements of a chained indexing access.
This matches behavior of 'cdata' in luajit and will provide a familiarity for developers switching over.

VM metatable lookups and `getmetatable` will look for the buffer object metatable first and then fall back to the global buffer metatable.
This preserves the existing buffer extensibility point for hosts.

`setmetatable` will still not be supported on buffer objects, the metatable reference is immutable.

Equality checks in the VM will call `__eq` for buffers similar to tables and userdata.

`__type` metatable key is ignored by `typeof`, just like it does for tables.
As before, only host is allowed to define type names.
This does not change how it behaved with a global host-provided metatable.

Other metamethods (`__add`/`__len`/...) work as expected and do not change from how they work today for buffers with global host-provided metatable.

Buffer structure size increase by the metatable aligns the data to a 16 byte boundary making it possible to store data which requires 16 byte alignment by the embedder.
This will make buffer data match the alignment guarantee of Luau userdata objects.

In order for the typechecker to understand buffers with attached metatables, we propose using the `setmetatable` type function:

`type Obj = setmetatable<buffer, { __index = { x: number, y: number, z: number } }>`

If there is already a metatable on a buffer type, `setmetatable` will fail.

## Forwards compatibility

A potential evolution of the buffer type might move the data from the buffer memory block to a separate location with a redirection.

Doing that will enable extending the buffer size without a major impact on performance (shrinking is a problem for range check eliminations).
It might also allow buffer views and slices.

In both cases, this change is compatible. The slice will reuse a section of the data but being a buffer it will have its own metatable.

This makes it possible to have a large memory buffer and sub-allocate buffer objects with attached custom behaviors.

## Drawbacks
Copy link
Contributor

@alexmccord alexmccord Dec 16, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't seem to mention anything about the performance characteristics of buffer vs table (even if deserialized from buffers) wrt algorithms that does heavy reads/writes? If I have an algorithm that is very heavy on reads/writes, it has to undergo some overhead to turn a few bytes of a buffer into a TValue (or vice versa) whereas tables skips this ser/de.

Copy link
Contributor

@alexmccord alexmccord Jan 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would like to also add another drawback not mentioned here: buffers doesn't let you store any pointer indirections for any other Luau objects, so you're unable to store nested buffers, strings, instances, tables, etc, limiting their usefulness as data structures.

Copy link
Contributor

@alexmccord alexmccord Jan 6, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And on the subject of the lack of pointer indirections: if you have a buffer of length 5, and you have a __newindex that accepts a string, you have to throw an error if the length of the string (minus the offset in the buffer) exceeds the buffer's capacity to store it. This makes it not ideal to allow the outside world to try and store strings in buffers without encapsulation.


This increases buffer size by 8 bytes, with the bigger impact on 0-8 byte buffers going from 16 to 24 bytes, with an overhead decreasing with larger sizes as before.
Copy link
Contributor

@alexmccord alexmccord Dec 16, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If each buffer needs a pointer to their own metatable, which is 1 word, and you need to know which buffer is host or user defined, then you need that 1 additional bit (and 7 bits padding) to store somewhere. And what happens if you need lua_newbufferdtor a la lua_newuserdatadtor (this feels inevitable)? That would incur yet another 1 word. As I understand this, it doesn't look like you can fit all of this in 8 bytes?

Copy link
Collaborator Author

@vegorov-rbx vegorov-rbx Dec 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is no need to separate host and user-defined buffers, implementation of the RFC doesn't have this extra bit.

As a side note, there are 8 bits available in the buffer padding today.

Starting from 64 bytes the increase is often free based on the current Luau allocator design.
In particular, sizes like 64/96/128/256 do not increase in internal allocation class.

This RFC also introduces a special evaluation rule for metamethod functions.
It is introduced for potential improvement in caching of operations, but might come at a surprise to users of deprecated environment modification functions.
While similar to the effects of function inlining Luau performs, it is an extra item to keep in mind.

## Alternatives

Instead of extending the buffer object with a limited set of functionality, we might pursue a new kind of object like Records were which can build internal field mappings for an easier optimization path.

Another possibility is some alternative way of specifying the fields that would support building both the right `__index` function and internal acceleration structures.