Add configurable email notification thresholds and lifecycle events

[namolnad]

Add configurable email notification thresholds and lifecycle events

Summary

This PR adds granular control over when error notification emails are sent, reducing noise while ensuring important errors don't go unnoticed. Instead of receiving an email for every single error occurrence, you can now configure notifications based on occurrence milestones, rate thresholds, and lifecycle events (resolved/reopened).

Closes #84

Motivation

In production environments, some errors occur frequently but are non-critical (e.g., external API timeouts that auto-retry). Receiving an email for every occurrence creates alert fatigue. This PR allows teams to:

• Get notified on the 1st, 10th, 100th, and 1000th occurrence to track error progression
• Receive alerts when errors spike (e.g., 30 occurrences in 5 minutes)
• Be notified when errors are resolved or reopened
• Reduce noise by disabling emails for non-critical errors while keeping database tracking

Features

1. Milestone-based notifications

Send emails only at specific occurrence counts with ordinalized subject lines:

SolidErrors.email_milestone_counts = [1, 10, 100, 1000]

Email subjects:

• 🔥 StandardError (1st occurrence)
• 🔥 StandardError (10th occurrence)
• 🔥 StandardError (100th occurrence)

Set to [] to disable count-based emails entirely, or leave as nil (default) for the current behavior (email every occurrence).

2. Rate threshold notifications

Get alerted when errors occur too frequently:

SolidErrors.email_rate_threshold_count = 30
SolidErrors.email_rate_threshold_window = 300 # 5 minutes in seconds

Sends one email when exactly 30 occurrences happen within a 5-minute window. Prevents spam by only triggering once when the threshold is crossed.

3. Lifecycle event notifications

Track when errors are resolved or reopened:

SolidErrors.email_on_resolved = true
SolidErrors.email_on_reopened = true

Email subjects:

• ✅ RESOLVED: StandardError - when an error is marked as resolved
• 🔄 REOPENED: StandardError - when a resolved error occurs again

Usage Examples

Example 1: High-traffic application

# config/environments/production.rb
config.solid_errors.send_emails = true
config.solid_errors.email_to = "team@example.com"

# Only alert on 1st occurrence and every 100 after that
SolidErrors.email_milestone_counts = [1, 100, 200, 300]

# Alert if we see 50 errors within 10 minutes
SolidErrors.email_rate_threshold_count = 50
SolidErrors.email_rate_threshold_window = 600

Example 2: Small team wanting full visibility

# Alert on key milestones and lifecycle changes
SolidErrors.email_milestone_counts = [1, 10, 50, 100]
SolidErrors.email_on_resolved = true
SolidErrors.email_on_reopened = true

Example 3: Disable count-based emails, only use rate threshold

# No emails for individual occurrences
SolidErrors.email_milestone_counts = []

# Only alert if errors spike
SolidErrors.email_rate_threshold_count = 30
SolidErrors.email_rate_threshold_window = 300

Backward Compatibility

✅ Fully backward compatible - all new settings default to nil or false, preserving existing behavior:

• email_milestone_counts = nil → emails sent for every occurrence (current behavior)
• email_rate_threshold_count = nil → no rate limiting
• email_on_resolved = false → no lifecycle emails
• email_on_reopened = false → no lifecycle emails

Existing applications require no configuration changes.

Implementation Details

• Occurrence count emails now include a trigger: :milestone parameter to customize subject lines
• Lifecycle emails reuse the existing error_occurred mailer with trigger: :resolved or trigger: :reopened
• Rate threshold uses a single database query with a time range filter
• All email sending respects the existing send_emails and email_to configuration

Testing

Tests cover:

• Default behavior (nil values)
• Milestone counting at various thresholds
• Rate threshold triggering and non-triggering scenarios
• Lifecycle event emails (resolved/reopened)
• Backward compatibility

Related

Addresses the use case described in #84 for milestone and lifecycle event notifications.