Add landscaping-assistant PrometheusRule alerts and dedicated Telegram channel #409

New issue

Closed

opened 2026-06-05 03:49:37 +00:00 by ldraney · 0 comments

ldraney commented

2026-06-05 03:49:37 +00:00

Owner

Type

Feature

Lineage

Parent: landscaping-assistant#95 (AlertManager cleanup)
Related: landscaping-assistant#17 (PrometheusRule alerts)

Repo

pal-e-platform

User Story

As a solo operator, I want landscaping-assistant alerts routed to a dedicated Telegram channel so I can monitor app health separately from platform noise.

Problem

landscaping-assistant exposes metrics via yabeda but has zero alerting rules. If the app throws 500s or goes down, there's no notification. All alerts also route to a single Telegram group — no per-app separation.

Solution

Add telegram-landscaping receiver with dedicated Telegram group chat_id
Add AlertManager route: namespace=landscaping-assistant → telegram-landscaping
Create PrometheusRule: ErrorRateHigh, LatencyHigh, PumaThreadSaturation, AppDown

Context

Default kube-prometheus-stack alerting rules were disabled in PR #408. The noise floor is now clean (11 firing alerts, all real). This is the first app-specific alerting — establishes the pattern for other services. Telegram group already created, bot added as admin, chat_id confirmed via API test.

File Targets

terraform/modules/monitoring/main.tf — receiver, route, PrometheusRule resource
terraform/modules/monitoring/variables.tf — new telegram_landscaping_chat_id variable
terraform/main.tf — pass new variable to module
terraform/variables.tf — root-level variable declaration

Test Expectations

All 4 alert rules visible in Prometheus after apply
Test message confirms routing to dedicated Telegram group
General channel continues receiving non-landscaping alerts

Constraints

Reuse existing bot token — only the chat_id differs
Recording rules unaffected
No changes to existing receivers or routes

Acceptance Criteria

Dedicated Telegram group receives landscaping-assistant alerts only
General Telegram group continues receiving all other alerts
All 4 alert rules visible in Prometheus
Terraform plan clean, apply successful

Checklist

Issue created
Branch created
PR submitted
Review passed
Merged and applied

landscaping-assistant#95
landscaping-assistant#17

### Type Feature ### Lineage - Parent: landscaping-assistant#95 (AlertManager cleanup) - Related: landscaping-assistant#17 (PrometheusRule alerts) ### Repo pal-e-platform ### User Story As a solo operator, I want landscaping-assistant alerts routed to a dedicated Telegram channel so I can monitor app health separately from platform noise. ### Problem landscaping-assistant exposes metrics via yabeda but has zero alerting rules. If the app throws 500s or goes down, there's no notification. All alerts also route to a single Telegram group — no per-app separation. ### Solution 1. Add `telegram-landscaping` receiver with dedicated Telegram group chat_id 2. Add AlertManager route: `namespace=landscaping-assistant` → `telegram-landscaping` 3. Create PrometheusRule: ErrorRateHigh, LatencyHigh, PumaThreadSaturation, AppDown ### Context Default kube-prometheus-stack alerting rules were disabled in PR #408. The noise floor is now clean (11 firing alerts, all real). This is the first app-specific alerting — establishes the pattern for other services. Telegram group already created, bot added as admin, chat_id confirmed via API test. ### File Targets - `terraform/modules/monitoring/main.tf` — receiver, route, PrometheusRule resource - `terraform/modules/monitoring/variables.tf` — new `telegram_landscaping_chat_id` variable - `terraform/main.tf` — pass new variable to module - `terraform/variables.tf` — root-level variable declaration ### Test Expectations - All 4 alert rules visible in Prometheus after apply - Test message confirms routing to dedicated Telegram group - General channel continues receiving non-landscaping alerts ### Constraints - Reuse existing bot token — only the chat_id differs - Recording rules unaffected - No changes to existing receivers or routes ### Acceptance Criteria - [ ] Dedicated Telegram group receives landscaping-assistant alerts only - [ ] General Telegram group continues receiving all other alerts - [ ] All 4 alert rules visible in Prometheus - [ ] Terraform plan clean, apply successful ### Checklist - [ ] Issue created - [ ] Branch created - [ ] PR submitted - [ ] Review passed - [ ] Merged and applied ### Related - landscaping-assistant#95 - landscaping-assistant#17