Reduce TTFX latency for common IMASdd operations by PrecompileTools#81
Reduce TTFX latency for common IMASdd operations by PrecompileTools#81
Conversation
- Move precompile workload to dedicated precompile_workload.jl - Add file I/O precompilation with safe mktempdir cleanup - Precompile REPL display methods using devnull output - Include equality, deepcopy operations in workload
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #81 +/- ##
==========================================
- Coverage 43.73% 43.72% -0.01%
==========================================
Files 13 14 +1
Lines 31230 31230
==========================================
- Hits 13657 13655 -2
- Misses 17573 17575 +2 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
Thanks @mgyoo86! This has been on my radar for a while. As you say, there are trade-offs, which for the average user is probably worthwhile, but it's not clear to me how many users vs developers we have. On top of that, we do release a new version of IMASdd every few weeks, so that precompile price gets paid for users when they upgrade as well. Maybe the paces of our releases will decrease @fredrikekre is working on some improvements to IMASdd that would hopefully help with precompilation and compilation time. I think we should put this on hold for a bit and have it be part of a more comprehensive improvement to compilation. An alternative might be to have the default maintain precompilation off and have users opt in with Preferences.jl for now. |
|
It looks like this shouldn't apply, but we should be careful that we don't invalidate any of this precompilation in IMAS.jl, like we used to in the expressions (see ProjectTorreyPines/IMAS.jl#276 and #77) @mgyoo86 After precompilation, please |
Summary
Adds comprehensive
@compile_workloadto significantly reduce TTFX (time to first execution) for common IMASdd operations. While this increases initial precompile (one-time cost) and load time (costs per session), it delivers dramatic improvements in runtime performance for frequently used functions.Performance Impact
One-Time (& per-session) Costs
using IMASdd)Runtime Performance Gains
hdf2imas()json2imas()imas2hdf()imas2json()deepcopy(dd)dd1==dd2Trade-off Analysis
Cost: +3 seconds per IMASdd loading
Benefit: Save 10~30 seconds on every file I/O operation
For typical workflows with I/O operations, the net time savings are substantial after the first operation.
Changes
src/precompile_workload.jlwith comprehensive workloadmktempdir()for safe temporary file cleanupdevnulloutput to avoid precompile clutterDeveloper Notes
Most developers/users rarely modify IMASdd, making the +100s precompile increase a true one-time installation cost. For active development, the workload can be disabled by
PrecompileTools.jlwithPreferences.jl