feat[next]: Add instrumentation package and user-defineable hooks

[egparedes]

This PR adds the instrumentation subpackage to gt4py.next containing the already existing performance metrics infrastructure and a new hook infrastructure to allow users to attach custom callbacks to different parts of the gt4py programs execution.

[Copilot]

Pull request overview

This PR introduces an instrumentation subpackage to gt4py.next containing performance metrics infrastructure and a new hook system for user-defined callbacks. The existing metrics module is moved from gt4py.next.metrics to gt4py.next.instrumentation.metrics and refactored to simplify the API by replacing the SourceHandler class with a more direct approach.

Changes:

• Adds new hook machinery supporting event and context hooks with signature validation and callback registration
• Refactors metrics collection to simplify source management (removes SourceHandler class)
• Creates three public hooks: program_call_hook, embedded_program_call_hook, and compile_variant_hook

Reviewed changes

Copilot reviewed 15 out of 15 changed files in this pull request and generated 9 comments.

Show a summary per file

File	Description
src/gt4py/next/instrumentation/__init__.py	New package initialization file
src/gt4py/next/instrumentation/_hook_machinery.py	Core hook implementation with EventHook and ContextHook classes
src/gt4py/next/instrumentation/hooks.py	Public hook exports for program and compilation callbacks
src/gt4py/next/instrumentation/metrics.py	Refactored metrics with simplified API (removed SourceHandler, renamed functions)
src/gt4py/next/otf/compiled_program.py	Integrated compile_variant_hook and updated metrics API calls
src/gt4py/next/ffront/decorator.py	Integrated program call hooks and updated metrics API calls
tests/next_tests/unit_tests/instrumentation_tests/__init__.py	Test package initialization
tests/next_tests/unit_tests/instrumentation_tests/test_hook_machinery.py	Comprehensive unit tests for hook machinery
tests/next_tests/integration_tests/feature_tests/instrumentation_tests/__init__.py	Integration test package initialization
tests/next_tests/integration_tests/feature_tests/instrumentation_tests/test_hooks.py	Integration tests for hooks with different backends
tests/next_tests/unit_tests/instrumentation_tests/test_metrics.py	Updated import path
tests/next_tests/integration_tests/feature_tests/ffront_tests/test_decorator.py	Updated import paths in mocks
src/gt4py/next/program_processors/runners/gtfn.py	Updated import path
src/gt4py/next/program_processors/runners/dace/workflow/translation.py	Updated import path
src/gt4py/next/program_processors/runners/dace/workflow/decoration.py	Updated import path

Comments suppressed due to low confidence (2)

src/gt4py/next/instrumentation/metrics.py:197

• Assigning a class directly to collect changes the API from metrics.collect() (function call) to metrics.collect() (class instantiation). While both syntactically work, this is a subtle API change. Consider documenting this or maintaining the function-based API for clarity by defining collect = CollectorContextManager as def collect() -> CollectorContextManager: return CollectorContextManager().
  src/gt4py/next/instrumentation/metrics.py:135
• The function name is_current_source_set could be more concise. Consider renaming to has_current_source or is_collecting for better readability, especially since the previous function was in_collection_mode() which was more intuitive.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.