Skip to main content

Alert Rules Configuration

Overview​

Alert rules determine which incoming alerts from your monitoring systems trigger automatic investigations. Configure rules to focus OpsWorker on the alerts that matter most to your team.

Setup​

1. Navigate to Alert Rules​

In the OpsWorker portal, go to Configurations → Alert Rules.

2. Create a Rule​

Click Create Rule and configure:

FieldDescriptionExample
Rule nameA descriptive name"Critical production alerts"
ClusterWhich cluster this rule applies toproduction-us-east
Namespace filterRegex pattern for Kubernetes namespacesprod-.*
SeverityAlert severity levels to matchcritical, warning
Label filtersKey-value matching on alert labelsteam=payments
Auto-investigateWhether matching alerts auto-trigger investigationsEnabled/Disabled

3. Enable Auto-Investigation​

Toggle Auto-investigate to automatically start investigations when matching alerts arrive. When disabled, matching alerts are recorded but not investigated.

4. Save and Test​

Save the rule, then verify it works:

  • Wait for a real alert that matches your filter criteria
  • Send a test alert from your monitoring system
  • Use the Test Integration button in cluster settings

Example Rules​

Investigate All Critical Alerts in Production​

  • Namespace: prod-.*
  • Severity: critical
  • Auto-investigate: Enabled

Investigate Payment Service Alerts​

  • Labels: service=payment-api
  • Severity: critical, warning
  • Auto-investigate: Enabled

Monitor Staging (No Auto-Investigation)​

  • Namespace: staging
  • Severity: critical
  • Auto-investigate: Disabled (signals recorded for visibility)

Combining Filters​

Filters within a rule are combined with AND logic. For example:

  • Namespace: prod-.* AND Severity: critical
  • This matches only critical alerts in production namespaces

Create multiple rules for OR logic (e.g., one rule for critical alerts, another for payment service alerts).

Best Practices​

  • Start narrow: Begin with critical alerts in production namespaces
  • Expand gradually: Add rules for warning severity and additional namespaces as confidence grows
  • Review regularly: Check the daily digest to see which alerts are being missed

Next Steps​