Apply Terraform state drift: 5+ merged PRs unapplied (3 alerts)

forgejo_admin commented

2026-03-18 17:04:02 +00:00

Owner

Lineage

plan-pal-e-platform → Platform Hardening

Repo

forgejo_admin/pal-e-platform

User Story

As a platform operator
I want merged Terraform changes applied to the live cluster
So that fixes actually take effect and the cluster matches IaC

Context

At least 5 merged PRs have not been applied to the live cluster, causing:

Postgres TargetDown — PR #95 added monitoring ingress to postgres NetworkPolicy, never applied
CNPG backup verify failures — PR #93 fixed the verify script's WAL check, never applied

Root cause is likely CI clone failures (Issue #107). If Woodpecker can't clone, the apply-on-merge pipeline never runs. Manual run of the updated backup script passes — pal-e-postgres WALs fresh, woodpecker correctly skipped.

File Targets

No code changes needed — the fixes are already merged
terraform/main.tf — verify no unexpected drift
terraform/network-policies.tf — postgres NP fix from PR #95

Files NOT to touch:

Don't modify any Terraform source — just apply what's already merged

Acceptance Criteria

tofu apply succeeds (manually or via CI)
Postgres metrics target UP in Prometheus
Next backup verify CronJob passes
Cluster state matches Terraform source

Test Expectations

tofu plan -lock=false shows only expected changes (no surprises)
After apply: kubectl get podmonitor -n postgres shows correct config
After apply: next scheduled cnpg-backup-verify job succeeds
probe_success{service="postgres"} == 1 or TargetDown alert clears

Constraints

Blocked by Issue #107 (TLS clone fix) for CI-driven apply
Can be manually applied with tofu apply -lock=false as interim
Run tofu plan first to verify no unexpected state drift
Do NOT apply blindly — review the plan output

Checklist

tofu plan reviewed
tofu apply succeeds
Postgres target UP
Backup verify passes
No unrelated changes

pal-e-platform — project board
Issue #107 — TLS clone fix (blocker for CI apply)
Issue #109 — umbrella alert cleanup
PR #93 — backup verify fix (merged, unapplied)
PR #95 — postgres NetworkPolicy fix (merged, unapplied)

### Lineage `plan-pal-e-platform` → Platform Hardening ### Repo `forgejo_admin/pal-e-platform` ### User Story As a platform operator I want merged Terraform changes applied to the live cluster So that fixes actually take effect and the cluster matches IaC ### Context At least 5 merged PRs have not been applied to the live cluster, causing: - **Postgres TargetDown** — PR #95 added monitoring ingress to postgres NetworkPolicy, never applied - **CNPG backup verify failures** — PR #93 fixed the verify script's WAL check, never applied Root cause is likely CI clone failures (Issue #107). If Woodpecker can't clone, the apply-on-merge pipeline never runs. Manual run of the updated backup script passes — pal-e-postgres WALs fresh, woodpecker correctly skipped. ### File Targets - No code changes needed — the fixes are already merged - `terraform/main.tf` — verify no unexpected drift - `terraform/network-policies.tf` — postgres NP fix from PR #95 Files NOT to touch: - Don't modify any Terraform source — just apply what's already merged ### Acceptance Criteria - [ ] `tofu apply` succeeds (manually or via CI) - [ ] Postgres metrics target UP in Prometheus - [ ] Next backup verify CronJob passes - [ ] Cluster state matches Terraform source ### Test Expectations - [ ] `tofu plan -lock=false` shows only expected changes (no surprises) - [ ] After apply: `kubectl get podmonitor -n postgres` shows correct config - [ ] After apply: next scheduled `cnpg-backup-verify` job succeeds - [ ] `probe_success{service="postgres"} == 1` or TargetDown alert clears ### Constraints - Blocked by Issue #107 (TLS clone fix) for CI-driven apply - Can be manually applied with `tofu apply -lock=false` as interim - Run `tofu plan` first to verify no unexpected state drift - Do NOT apply blindly — review the plan output ### Checklist - [ ] `tofu plan` reviewed - [ ] `tofu apply` succeeds - [ ] Postgres target UP - [ ] Backup verify passes - [ ] No unrelated changes ### Related - `pal-e-platform` — project board - Issue #107 — TLS clone fix (blocker for CI apply) - Issue #109 — umbrella alert cleanup - PR #93 — backup verify fix (merged, unapplied) - PR #95 — postgres NetworkPolicy fix (merged, unapplied)

forgejo_admin referenced this issue

2026-03-18 17:11:24 +00:00

bug: Woodpecker CI clone fails with TLS EOF on Tailscale funnel #107

forgejo_admin commented

2026-03-18 18:10:28 +00:00

Author

Owner

Scope Review: READY

Review note: review-192-2026-03-18
Scope is complete — all template sections present, both file targets verified in codebase, blocker Issue #107 is now closed. Agent can execute as written.

Issue #107 blocker resolved (closed, board item done) — CI apply should work without manual workaround
PR #93 WAL skip fix and PR #95 monitoring ingress fix both confirmed merged in main
Note: tofu plan will show drift from ~7 merged PRs (not just the 2 named), but acceptance criteria correctly require plan review before apply

## Scope Review: READY Review note: `review-192-2026-03-18` Scope is complete — all template sections present, both file targets verified in codebase, blocker Issue #107 is now closed. Agent can execute as written. - Issue #107 blocker resolved (closed, board item done) — CI apply should work without manual workaround - PR #93 WAL skip fix and PR #95 monitoring ingress fix both confirmed merged in main - Note: `tofu plan` will show drift from ~7 merged PRs (not just the 2 named), but acceptance criteria correctly require plan review before apply

forgejo_admin commented

2026-03-18 18:27:40 +00:00

Author

Owner

Scope Review: READY

Review note: review-192-2026-03-18
Scope is complete and verified. All template sections present, both file targets confirmed in codebase, blocker Issue #107 is now closed.

All 10 template-issue sections present
terraform/network-policies.tf line 152 confirms PR #95 monitoring ingress merged
terraform/main.tf line 2265 confirms PR #93 backup verify fix merged
Issue #107 blocker resolved (closed 2026-03-18) — CI apply-on-merge should function
Acceptance criteria are all agent-verifiable with real test commands

## Scope Review: READY Review note: `review-192-2026-03-18` Scope is complete and verified. All template sections present, both file targets confirmed in codebase, blocker Issue #107 is now closed. - All 10 template-issue sections present - `terraform/network-policies.tf` line 152 confirms PR #95 monitoring ingress merged - `terraform/main.tf` line 2265 confirms PR #93 backup verify fix merged - Issue #107 blocker resolved (closed 2026-03-18) — CI apply-on-merge should function - Acceptance criteria are all agent-verifiable with real test commands

forgejo_admin referenced this issue

2026-03-18 19:01:50 +00:00

fix: add monitoring ingress to Keycloak NetworkPolicy + use internal probe URL #117

forgejo_admin referenced this issue

2026-03-18 19:02:10 +00:00

fix: add monitoring ingress to Keycloak NetworkPolicy + use internal probe URL #117

forgejo_admin referenced this issue from a commit

2026-03-18 19:49:28 +00:00

fix: override clone step to use internal Forgejo URL

forgejo_admin referenced this issue from a pull request that will close it,

2026-03-18 19:49:51 +00:00

fix: override clone step to use internal Forgejo URL #118