Skip to content

perf: regexp2 pretokenizer, pooled buffers, offset-free fast encode path#80

Open
aeryncaen wants to merge 2 commits intosugarme:masterfrom
aeryncaen:perf/qwen-pretokenizer-regexp2
Open

perf: regexp2 pretokenizer, pooled buffers, offset-free fast encode path#80
aeryncaen wants to merge 2 commits intosugarme:masterfrom
aeryncaen:perf/qwen-pretokenizer-regexp2

Commits

Commits on Feb 23, 2026