Skip to content

Optimize runtime complexity of linter when passing in multiple snippets #725

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
johnbartholomew merged 2 commits into from
Jan 27, 2026
Merged

Optimize runtime complexity of linter when passing in multiple snippets #725

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
8893d8e
Optimize runtime complexity of linter when passing in multiple snippets.
netomi Aug 13, 2023
23a013c
linter: rely on ImportAST internal caching, isolate variables for eac…
johnbartholomew Jan 27, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 14 additions & 23 deletions linter/linter.go
View file
Open in desktop
Copy link
Collaborator

@johnbartholomew johnbartholomew Jan 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking through the code history, support for linting multiple files in a single invocation was added in #545, and the review at the time noted that it doesn't really take advantage of being able to collect and process the expression graphs for all the files together. But the review did not note that just partially processing the files together would itself create a performance problem.

It seems like from there we can go in one of two directions: either split processing apart more, so each listed input file is processed mostly independently of the others (which is the direction this PR goes in), or make a deeper change in the linter expression/type graph processing to more efficiently process all the input files with a single expression/type graph. Since the latter is obviously a more invasive change I'm happy to stick with this fix. It should be at least no worse than running the lint command entirely separately on each individual input file.

I do have a few smaller structural comments, added below.

Original file line number Diff line number Diff line change
Expand Up @@ -45,39 +45,30 @@ type nodeWithLocation struct {

// Lint analyses a node and reports any issues it encounters to an error writer.
func lint(vm *jsonnet.VM, nodes []nodeWithLocation, errWriter *ErrorWriter) {
roots := make(map[string]ast.Node)
for _, node := range nodes {
roots[node.path] = node.node
}
for _, node := range nodes {
getImports(vm, node, roots, errWriter)
}

variablesInFile := make(map[string]common.VariableInfo)

std := common.Variable{
Name: "std",
Occurences: nil,
VariableKind: common.VarStdlib,
}

findVariables := func(node nodeWithLocation) *common.VariableInfo {
return variables.FindVariables(node.node, variables.Environment{"std": &std, "$std": &std})
}
stdEnv := variables.Environment{"std": &std, "$std": &std}

for importedPath, rootNode := range roots {
variablesInFile[importedPath] = *findVariables(nodeWithLocation{rootNode, importedPath})
}
for _, node := range nodes {
roots := make(map[string]ast.Node)
roots[node.path] = node.node
getImports(vm, node, roots, errWriter)

vars := make(map[string]map[ast.Node]*common.Variable)
for importedPath, info := range variablesInFile {
vars[importedPath] = info.VarAt
}
variablesInFile := make(map[string]common.VariableInfo)
for importedPath, rootNode := range roots {
variablesInFile[importedPath] = *variables.FindVariables(rootNode, stdEnv)
}

for _, node := range nodes {
variableInfo := findVariables(node)
vars := make(map[string]map[ast.Node]*common.Variable)
for importedPath, info := range variablesInFile {
vars[importedPath] = info.VarAt
}

for _, v := range variableInfo.Variables {
for _, v := range variablesInFile[node.path].Variables {
if len(v.Occurences) == 0 && v.VariableKind == common.VarRegular && v.Name != "$" {
errWriter.writeError(vm, errors.MakeStaticError("Unused variable: "+string(v.Name), v.LocRange))
}
Expand Down