-
Notifications
You must be signed in to change notification settings - Fork 3
Description
When constructing Templates, we should check that the variables in the docstring match the Template signature. The string should be a valid format string that refers to valid lexical variables in the Template body's lexical scope.
For example, the following should always be disallowed when author is undefined:
@Template.define
def write_poem(topic: str) -> str:
"""Write a poem about {topic} by {author}"""
raise NotHandledWhen author is a lexical variable in an enclosing scope, we could allow it to be referenced, or we could be strict and require Templates to refer only to arguments:
def poet(author: str) -> Template[[str], str]:
@Template.define
def write_poem(topic: str) -> str:
"""Write a poem about {topic} by {author}"""
raise NotHandled
return write_poemThese checks could be implemented by attempting to parse the template according to the official format string mini-language specification, and by attempting to substitute dummy values into variable names via .format(**self.__signature__.bind(*dummy_args, **dummy_kwargs).arguments)
Note: Python 3.14 adds a new module string.templatelib with a richer backwards-incompatible template string semantics; if we choose to support that in our prompt templates, which is out of scope for this issue, we should do similar validation.