SDCICD-1733: [WIP] Add KrknAI chaos test results aggregator

[varunraokadaparthi]

Krkn Aggregator:

Introduces a dedicated aggregator for KrknAI chaos testing results that reduces context for LLM analysis by parsing key summary files and excluding verbose cluster data.

Key features:

• Parses all.csv and health_check_report.csv to extract scenario results
• Extracts minimal config summary immune to key reordering from updateKrknConfig
• Stores absolute paths for log artifacts enabling LLM tool access
• Provides top N scenarios (by fitness), failed scenarios, and health check metrics

Files:

• internal/aggregator/krknai.go: Aggregator implementation
• internal/aggregator/krknai_test.go: Unit tests with 8 test cases

Krkn Engine:

Introduces a new analysis mode for KrknAI chaos test results alongside
the existing CI failure analysis. The engine uses LLM to analyze chaos
scenarios and provide cluster resilience recommendations.

Changes:

• internal/analysisengine/krknai.go: KrknAI analysis engine that
  orchestrates aggregation, prompt rendering, LLM analysis, and result
  reporting for chaos test results
• internal/llm/tools/tools.go: Updated tool registry to accept log
  artifacts directly, enabling read_file tool to access scenario files
• internal/llm/tools/read_file_test.go: Updated tests to use
  []aggregator.LogEntry instead of *aggregator.AggregatedData
• internal/prompts/templates/krknai.yaml: A dummy prompt template
  for chaos engineering analysis with JSON response schema

The KrknAI engine:

• Collects results via KrknAIAggregator
• Renders prompts with scenario summaries and artifact paths
• Provides LLM with read_file tool for deep-dive analysis
• Writes YAML summary to results directory
• Supports Slack notifications

Co-authored-by: Cursor AIcursor@cursor.sh

[openshift-ci-robot]

@varunraokadaparthi: This pull request references SDCICD-1733 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the sub-task to target the "4.22.0" version, but no target version was set.

Details

In response to this:

Introduces a dedicated aggregator for KrknAI chaos testing results that reduces context for LLM analysis by parsing key summary files and excluding verbose cluster data.

Key features:

• Parses all.csv and health_check_report.csv to extract scenario results
• Extracts minimal config summary immune to key reordering from updateKrknConfig
• Stores absolute paths for log artifacts enabling LLM tool access
• Provides top N scenarios (by fitness), failed scenarios, and health check metrics

Files:

• internal/aggregator/krknai.go: Aggregator implementation
• internal/aggregator/krknai_test.go: Unit tests with 8 test cases

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: varunraokadaparthi

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• OWNERS [varunraokadaparthi]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[openshift-ci-robot]

@varunraokadaparthi: This pull request references SDCICD-1733 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the sub-task to target the "4.22.0" version, but no target version was set.

Details

In response to this:

Introduces a dedicated aggregator for KrknAI chaos testing results that reduces context for LLM analysis by parsing key summary files and excluding verbose cluster data.

Key features:

• Parses all.csv and health_check_report.csv to extract scenario results
• Extracts minimal config summary immune to key reordering from updateKrknConfig
• Stores absolute paths for log artifacts enabling LLM tool access
• Provides top N scenarios (by fitness), failed scenarios, and health check metrics

Files:

• internal/aggregator/krknai.go: Aggregator implementation
• internal/aggregator/krknai_test.go: Unit tests with 8 test cases

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-ci-robot]

@varunraokadaparthi: This pull request references SDCICD-1733 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the sub-task to target the "4.22.0" version, but no target version was set.

Details

In response to this:

Krkn Aggregator:

Introduces a dedicated aggregator for KrknAI chaos testing results that reduces context for LLM analysis by parsing key summary files and excluding verbose cluster data.

Key features:

• Parses all.csv and health_check_report.csv to extract scenario results
• Extracts minimal config summary immune to key reordering from updateKrknConfig
• Stores absolute paths for log artifacts enabling LLM tool access
• Provides top N scenarios (by fitness), failed scenarios, and health check metrics

Files:

• internal/aggregator/krknai.go: Aggregator implementation
• internal/aggregator/krknai_test.go: Unit tests with 8 test cases

Krkn Engine:

Introduces a new analysis mode for KrknAI chaos test results alongside
the existing CI failure analysis. The engine uses LLM to analyze chaos
scenarios and provide cluster resilience recommendations.

Changes:

• internal/analysisengine/krknai.go: KrknAI analysis engine that
  orchestrates aggregation, prompt rendering, LLM analysis, and result
  reporting for chaos test results
• internal/llm/tools/tools.go: Updated tool registry to accept log
  artifacts directly, enabling read_file tool to access scenario files
• internal/llm/tools/read_file_test.go: Updated tests to use
  []aggregator.LogEntry instead of *aggregator.AggregatedData
• internal/prompts/templates/krknai.yaml: A dummy prompt template
  for chaos engineering analysis with JSON response schema

The KrknAI engine:

• Collects results via KrknAIAggregator
• Renders prompts with scenario summaries and artifact paths
• Provides LLM with read_file tool for deep-dive analysis
• Writes YAML summary to results directory
• Supports Slack notifications

Co-authored-by: Cursor AIcursor@cursor.sh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-ci]

@varunraokadaparthi: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

[varunraokadaparthi]

/hold

[ritmun]

Can the specific implementation go in krkai package ?

[varunraokadaparthi]

@ritmun You mean in internal/aggregator/krkn/krknai.go?