Intern import details/metadata line contents#185
Intern import details/metadata line contents#185seddonym merged 4 commits intopython-grimp:masterfrom
Conversation
CodSpeed Performance ReportMerging #185 will degrade performances by 46.5%Comparing Summary
Benchmarks breakdown
|
Else the copy benchmark isn't really fair
70e0a5c to
03dfe9f
Compare
I'd gone to all the trouble to remove strings from the graph in the module names, but then forgot about all the strings in the import details! This gives a signifcant boost to the performance of copying the graph.
03dfe9f to
bddb594
Compare
| line_number=1, | ||
| line_contents=f"import {imported}", |
There was a problem hiding this comment.
This change introduces a regression in the benchmark for test_copy_graph, but this is reasonable since now the graph object is much bigger!
rust/src/graph/graph_manipulation.rs
Outdated
| importer: ModuleToken, | ||
| imported: ModuleToken, | ||
| line_number: usize, | ||
| line_number: u16, |
There was a problem hiding this comment.
This gives a small but measurable performance benefit. I think people aren't really having import statements past line 65535 😅
There was a problem hiding this comment.
I agree with the change - but I can believe there are modules out there in the wild that get that big. How would Grimp behave if it did encounter such a module?
Ideally we should handle it gracefully, but as long as we have the new behaviour captured in a ticket, and the change mentioned in the CHANGELOG, I'm happy.
There was a problem hiding this comment.
How would Grimp behave if it did encounter such a module?
Yes maybe it's worth a bit of thought.
It's quite a micro-optimisation anyways, so I think instead I'll just opt to use u32 here instead - I'm near certain that nobody will overflow that!
There was a problem hiding this comment.
(With u32 I wouldn't update the changelog)
There was a problem hiding this comment.
I'm near certain that nobody will overflow that!
Agreed 😄
seddonym
left a comment
There was a problem hiding this comment.
Good stuff - I think the change to line_number needs at least a mention in the CHANGELOG though, would you mind adding that?
rust/src/graph/graph_manipulation.rs
Outdated
| importer: ModuleToken, | ||
| imported: ModuleToken, | ||
| line_number: usize, | ||
| line_number: u16, |
There was a problem hiding this comment.
I agree with the change - but I can believe there are modules out there in the wild that get that big. How would Grimp behave if it did encounter such a module?
Ideally we should handle it gracefully, but as long as we have the new behaviour captured in a ticket, and the change mentioned in the CHANGELOG, I'm happy.
This should be enough!
bddb594 to
ae3ea3d
Compare
In this PR:
test_copy_graphbenchmark much more realistic. This does lead to a regression in the benchmarks, but this regression is expected since now the graph object is much bigger.Since change 1. gives a performance regression, as compared with the benchmark on master, here are the benchmark results run locally comparing the 1st commit (add line contents to large graph fixture) with the last commit (interned line contents). This shows that the changes in this PR gives ~2.5x improvement to the graph copy.