Critical: Migrate basketball-api Postgres to CNPG shared cluster #187

New issue

Closed

opened 2026-03-27 00:36:33 +00:00 by forgejo_admin · 6 comments

forgejo_admin commented 2026-03-27 00:36:33 +00:00

Owner

Copy link

Type

Bug

Lineage

standalone — discovered during #184 CI investigation. Data safety audit revealed basketball-api Postgres has zero backup coverage.

Repo

forgejo_admin/pal-e-platform (CNPG cluster config) + forgejo_admin/pal-e-deployments (basketball-api kustomize) + forgejo_admin/basketball-api (connection string)

User Story

story:WS-S5 As superadmin, I want basketball-api's database on the CNPG shared cluster so that player data has daily backups, WAL archiving, and automated restore verification — because right now 76 players, 6 signed contracts, 68 registrations, and 16 jersey orders sit on a single local-path PVC with zero backup coverage.

What Broke

Nothing is broken yet — this is a ticking time bomb. The basketball-api Postgres is a standalone Deployment pod with:

• local-path PVC on archbox (single node, single disk)
• PV reclaim policy: Delete — if PVC is deleted, data is gone
• No automated backups — no Barman, no CronJob, no pg_dump, nothing
• No replication — single instance
• No point-in-time recovery — if data corrupts, we're stuck
• No restore verification — we'd never know backup was broken until we needed it

Meanwhile, the CNPG cluster (pal-e-postgres-1) has daily Barman → MinIO backups, WAL archiving, gzip compression, 7-day retention, and a daily cnpg-backup-verify CronJob that proves restores work.

Architecture

BEFORE (current — fragile):
┌─────────────────────────────┐
│ basketball-api namespace    │
│                             │
│  basketball-api pod         │
│       │ DATABASE_HOST=      │
│       │ postgres.basketball │
│       ▼ -api.svc            │
│  postgres Deployment        │
│       │                     │
│       ▼                     │
│  postgres-data PVC          │
│  (local-path, 1Gi,         │
│   reclaim=Delete)           │
│  ⚠️ NO BACKUPS              │
│  ⚠️ NO REPLICATION          │
│  ⚠️ NO WAL ARCHIVE          │
└─────────────────────────────┘

AFTER (target — enterprise):
┌─────────────────────────────┐
│ basketball-api namespace    │
│                             │
│  basketball-api pod         │
│       │ DATABASE_HOST=      │
│       │ pal-e-postgres-rw.  │
│       │ postgres.svc        │
│       │ DATABASE_NAME=      │
│       │ basketball          │
│       ▼                     │
└───────┼─────────────────────┘
        │
┌───────▼─────────────────────┐
│ postgres namespace (CNPG)   │
│                             │
│  pal-e-postgres-1           │
│  ├─ database: basketball    │
│  ├─ user: basketball        │
│  ├─ Barman → MinIO daily    │
│  ├─ WAL archive (gzip)      │
│  ├─ 7-day retention         │
│  └─ cnpg-backup-verify      │
│     (daily CronJob)         │
└─────────────────────────────┘

Repro Steps

1. kubectl get pv | grep basketball → reclaimPolicy: Delete
2. kubectl get backups -n basketball-api → no resources found
3. kubectl get scheduledbackup -n basketball-api → no resources found
4. Compare with: kubectl get backups -n postgres → 9 daily backups, all completed

Expected Behavior

Basketball-api database should have the same backup coverage as pal-e-docs: daily Barman snapshots, WAL archiving, 7-day retention, automated restore verification.

Environment

• Current basketball-api DB: PostgreSQL 16.12 (Alpine), standalone Deployment, basketball-api namespace
• CNPG cluster: PostgreSQL 17.4 (Debian), postgres namespace, Barman → MinIO
• DB size: 9 MB (trivial migration)
• Version gap: PG 16 → PG 17 (major version — requires pg_dump/pg_restore, not streaming)

File Targets

• terraform/main.tf or CNPG cluster manifest — add basketball database + user to shared cluster
• ~/pal-e-deployments/basketball-api/ — update kustomize overlay: new DATABASE_HOST, remove standalone postgres Deployment
• ~/basketball-api/src/basketball_api/config.py — verify connection string works with new host
• ~/basketball-api/k8s/deployment.yaml — update DATABASE_HOST env var

Migration Steps

1. Create basketball database + user on CNPG cluster
2. pg_dump from standalone pod → pg_restore into CNPG (9MB, seconds)
3. Update basketball-api deployment to point at CNPG host (pal-e-postgres-rw.postgres.svc.cluster.local)
4. Verify app connects and data is intact
5. Run one backup cycle + verify
6. Delete old standalone postgres Deployment + PVC
7. Verify cnpg-backup-verify CronJob covers the new database

Test Expectations

• curl basketball-api.tail5b443a.ts.net/health returns 200
• SELECT count(*) FROM players returns 76 (or current count)
• SELECT count(*) FROM orders WHERE status = 'paid' matches pre-migration
• kubectl get backups -n postgres shows next daily backup includes basketball data
• cnpg-backup-verify CronJob completes successfully

Acceptance Criteria

• basketball database exists on CNPG shared cluster
• basketball-api connects to CNPG, all endpoints functional
• Daily Barman backup covers basketball database
• cnpg-backup-verify CronJob validates basketball data restores correctly
• Old standalone postgres Deployment + PVC removed
• Zero downtime (point basketball-api at CNPG, then remove old)

Constraints

• PG 16 → PG 17 major version: must use pg_dump/pg_restore, not pg_basebackup
• 9MB database — migration is seconds, but validate row counts before/after
• basketball-api must have zero downtime during cutover
• Do NOT delete old PVC until new setup is verified end-to-end
• Network policy in postgres namespace must allow traffic from basketball-api namespace

Related

• forgejo_admin/pal-e-platform#184 — CI blocker (triggered this discovery)
• project-westside-basketball — project this affects
• CNPG backup config: terraform/main.tf (Barman → MinIO, 7-day retention)
• sop-postgres-restore — existing restore SOP for CNPG

### Type Bug ### Lineage standalone — discovered during #184 CI investigation. Data safety audit revealed basketball-api Postgres has zero backup coverage. ### Repo `forgejo_admin/pal-e-platform` (CNPG cluster config) + `forgejo_admin/pal-e-deployments` (basketball-api kustomize) + `forgejo_admin/basketball-api` (connection string) ### User Story `story:WS-S5` As superadmin, I want basketball-api's database on the CNPG shared cluster so that player data has daily backups, WAL archiving, and automated restore verification — because right now 76 players, 6 signed contracts, 68 registrations, and 16 jersey orders sit on a single local-path PVC with zero backup coverage. ### What Broke Nothing is broken yet — this is a **ticking time bomb**. The basketball-api Postgres is a standalone Deployment pod with: - `local-path` PVC on archbox (single node, single disk) - PV reclaim policy: `Delete` — if PVC is deleted, data is gone - **No automated backups** — no Barman, no CronJob, no pg_dump, nothing - **No replication** — single instance - **No point-in-time recovery** — if data corrupts, we're stuck - **No restore verification** — we'd never know backup was broken until we needed it Meanwhile, the CNPG cluster (`pal-e-postgres-1`) has daily Barman → MinIO backups, WAL archiving, gzip compression, 7-day retention, and a daily `cnpg-backup-verify` CronJob that proves restores work. ### Architecture ``` BEFORE (current — fragile): ┌─────────────────────────────┐ │ basketball-api namespace │ │ │ │ basketball-api pod │ │ │ DATABASE_HOST= │ │ │ postgres.basketball │ │ ▼ -api.svc │ │ postgres Deployment │ │ │ │ │ ▼ │ │ postgres-data PVC │ │ (local-path, 1Gi, │ │ reclaim=Delete) │ │ ⚠️ NO BACKUPS │ │ ⚠️ NO REPLICATION │ │ ⚠️ NO WAL ARCHIVE │ └─────────────────────────────┘ AFTER (target — enterprise): ┌─────────────────────────────┐ │ basketball-api namespace │ │ │ │ basketball-api pod │ │ │ DATABASE_HOST= │ │ │ pal-e-postgres-rw. │ │ │ postgres.svc │ │ │ DATABASE_NAME= │ │ │ basketball │ │ ▼ │ └───────┼─────────────────────┘ │ ┌───────▼─────────────────────┐ │ postgres namespace (CNPG) │ │ │ │ pal-e-postgres-1 │ │ ├─ database: basketball │ │ ├─ user: basketball │ │ ├─ Barman → MinIO daily │ │ ├─ WAL archive (gzip) │ │ ├─ 7-day retention │ │ └─ cnpg-backup-verify │ │ (daily CronJob) │ └─────────────────────────────┘ ``` ### Repro Steps 1. `kubectl get pv | grep basketball` → `reclaimPolicy: Delete` 2. `kubectl get backups -n basketball-api` → no resources found 3. `kubectl get scheduledbackup -n basketball-api` → no resources found 4. Compare with: `kubectl get backups -n postgres` → 9 daily backups, all completed ### Expected Behavior Basketball-api database should have the same backup coverage as pal-e-docs: daily Barman snapshots, WAL archiving, 7-day retention, automated restore verification. ### Environment - Current basketball-api DB: PostgreSQL 16.12 (Alpine), standalone Deployment, `basketball-api` namespace - CNPG cluster: PostgreSQL 17.4 (Debian), `postgres` namespace, Barman → MinIO - DB size: **9 MB** (trivial migration) - Version gap: PG 16 → PG 17 (major version — requires pg_dump/pg_restore, not streaming) ### File Targets - `terraform/main.tf` or CNPG cluster manifest — add `basketball` database + user to shared cluster - `~/pal-e-deployments/basketball-api/` — update kustomize overlay: new DATABASE_HOST, remove standalone postgres Deployment - `~/basketball-api/src/basketball_api/config.py` — verify connection string works with new host - `~/basketball-api/k8s/deployment.yaml` — update DATABASE_HOST env var ### Migration Steps 1. Create `basketball` database + user on CNPG cluster 2. `pg_dump` from standalone pod → `pg_restore` into CNPG (9MB, seconds) 3. Update basketball-api deployment to point at CNPG host (`pal-e-postgres-rw.postgres.svc.cluster.local`) 4. Verify app connects and data is intact 5. Run one backup cycle + verify 6. Delete old standalone postgres Deployment + PVC 7. Verify `cnpg-backup-verify` CronJob covers the new database ### Test Expectations - `curl basketball-api.tail5b443a.ts.net/health` returns 200 - `SELECT count(*) FROM players` returns 76 (or current count) - `SELECT count(*) FROM orders WHERE status = 'paid'` matches pre-migration - `kubectl get backups -n postgres` shows next daily backup includes basketball data - `cnpg-backup-verify` CronJob completes successfully ### Acceptance Criteria - [ ] `basketball` database exists on CNPG shared cluster - [ ] basketball-api connects to CNPG, all endpoints functional - [ ] Daily Barman backup covers basketball database - [ ] `cnpg-backup-verify` CronJob validates basketball data restores correctly - [ ] Old standalone postgres Deployment + PVC removed - [ ] Zero downtime (point basketball-api at CNPG, then remove old) ### Constraints - PG 16 → PG 17 major version: must use pg_dump/pg_restore, not pg_basebackup - 9MB database — migration is seconds, but validate row counts before/after - basketball-api must have zero downtime during cutover - Do NOT delete old PVC until new setup is verified end-to-end - Network policy in postgres namespace must allow traffic from basketball-api namespace ### Related - `forgejo_admin/pal-e-platform#184` — CI blocker (triggered this discovery) - `project-westside-basketball` — project this affects - CNPG backup config: `terraform/main.tf` (Barman → MinIO, 7-day retention) - `sop-postgres-restore` — existing restore SOP for CNPG

forgejo_admin commented 2026-03-27 00:45:55 +00:00

Author

Owner

Copy link

Scope Review: NEEDS_REFINEMENT

Review note: review-417-2026-03-26

Well-scoped ticket with complete traceability and thorough technical detail, but four issues need resolution before it's ready for execution:

• CNPG cluster definition location unknown -- the pal-e-postgres shared cluster is not defined in terraform (either pal-e-platform or pal-e-services). Ticket must clarify where the cluster manifest lives and how basketball database + user get created on an existing cluster.
• Network policy change missing from scope -- terraform/network-policies.tf postgres namespace policy does not allow basketball-api ingress. This is mentioned in Constraints but absent from File Targets and Acceptance Criteria.
• File target paths incorrect -- ~/pal-e-deployments/basketball-api/ should be overlays/basketball-api/prod/. The ~/basketball-api/k8s/deployment.yaml is not the ArgoCD deploy target (kustomize overlay is).
• Backup verify AC misleading -- cnpg-backup-verify CronJob checks WAL freshness by prefix, not per-database restore validation. Reword to match actual behavior.

Blast radius: mcd-tracker and pal-e-mail have the identical standalone postgres:16-alpine pattern with zero backup coverage. Discovered scope -- separate tickets needed.

## Scope Review: NEEDS_REFINEMENT Review note: `review-417-2026-03-26` Well-scoped ticket with complete traceability and thorough technical detail, but four issues need resolution before it's ready for execution: - **CNPG cluster definition location unknown** -- the `pal-e-postgres` shared cluster is not defined in terraform (either pal-e-platform or pal-e-services). Ticket must clarify where the cluster manifest lives and how `basketball` database + user get created on an existing cluster. - **Network policy change missing from scope** -- `terraform/network-policies.tf` postgres namespace policy does not allow `basketball-api` ingress. This is mentioned in Constraints but absent from File Targets and Acceptance Criteria. - **File target paths incorrect** -- `~/pal-e-deployments/basketball-api/` should be `overlays/basketball-api/prod/`. The `~/basketball-api/k8s/deployment.yaml` is not the ArgoCD deploy target (kustomize overlay is). - **Backup verify AC misleading** -- `cnpg-backup-verify` CronJob checks WAL freshness by prefix, not per-database restore validation. Reword to match actual behavior. **Blast radius:** `mcd-tracker` and `pal-e-mail` have the identical standalone postgres:16-alpine pattern with zero backup coverage. Discovered scope -- separate tickets needed.

forgejo_admin referenced this issue 2026-03-27 00:48:31 +00:00

Critical: Migrate mcd-tracker Postgres to CNPG shared cluster #189

forgejo_admin referenced this issue 2026-03-27 00:48:32 +00:00

Critical: Migrate pal-e-mail Postgres to CNPG shared cluster #190

forgejo_admin commented 2026-03-27 00:48:58 +00:00

Author

Owner

Copy link

Refinement (post review-417-2026-03-26)

Addressing 4 review findings + 1 critical discovery.

Critical discovery: CNPG cluster manifest is orphaned

The pal-e-postgres CNPG cluster was removed from terraform/main.tf on branch 16-remove-app-level-cnpg-resources-from-pla (commit c50f013) as part of architectural separation — it was supposed to move to pal-e-services but never landed there. The cluster is running in prod but has no manifest in any repo.

This means:

• We can't modify the cluster spec via IaC (adding databases, changing config)
• If the cluster is deleted, we'd have to reconstruct the manifest from memory
• Adding basketball database requires either: (a) re-creating the cluster manifest in pal-e-services, or (b) applying a raw kubectl command

Pre-requisite: Re-establish the CNPG cluster manifest in pal-e-services before this ticket can proceed.

Fixes from review

1. CNPG cluster location — documented above. Need to re-create manifest first.
2. Network policy — terraform/network-policies.tf:175-179 must add basketball-api namespace to the postgres ingress allow list. Adding to File Targets + AC.
3. File target paths corrected:
   • ~/pal-e-deployments/overlays/basketball-api/prod/ (not ~/basketball-api/k8s/)
   • terraform/network-policies.tf (new — network policy update)
4. Backup verification AC reworded — CronJob checks WAL object freshness, not per-database restore.

Discovered scope (filed)

• #189 — mcd-tracker same vulnerability (standalone postgres, zero backups)
• #190 — pal-e-mail same vulnerability (standalone postgres, zero backups)
• Orphaned CNPG manifest — needs its own ticket on pal-e-services before any migration can happen

## Refinement (post review-417-2026-03-26) Addressing 4 review findings + 1 critical discovery. ### Critical discovery: CNPG cluster manifest is orphaned The `pal-e-postgres` CNPG cluster was removed from `terraform/main.tf` on branch `16-remove-app-level-cnpg-resources-from-pla` (commit `c50f013`) as part of architectural separation — it was supposed to move to `pal-e-services` but **never landed there**. The cluster is running in prod but has no manifest in any repo. This means: - We can't modify the cluster spec via IaC (adding databases, changing config) - If the cluster is deleted, we'd have to reconstruct the manifest from memory - Adding `basketball` database requires either: (a) re-creating the cluster manifest in pal-e-services, or (b) applying a raw `kubectl` command **Pre-requisite:** Re-establish the CNPG cluster manifest in pal-e-services before this ticket can proceed. ### Fixes from review 1. **CNPG cluster location** — documented above. Need to re-create manifest first. 2. **Network policy** — `terraform/network-policies.tf:175-179` must add `basketball-api` namespace to the postgres ingress allow list. Adding to File Targets + AC. 3. **File target paths corrected:** - `~/pal-e-deployments/overlays/basketball-api/prod/` (not `~/basketball-api/k8s/`) - `terraform/network-policies.tf` (new — network policy update) 4. **Backup verification AC reworded** — CronJob checks WAL object freshness, not per-database restore. ### Discovered scope (filed) - #189 — mcd-tracker same vulnerability (standalone postgres, zero backups) - #190 — pal-e-mail same vulnerability (standalone postgres, zero backups) - Orphaned CNPG manifest — needs its own ticket on pal-e-services before any migration can happen

forgejo_admin commented 2026-03-27 07:47:16 +00:00

Author

Owner

Copy link

Refinement v2 (post review-417-2026-03-26 + CNPG manifest landed)

Prereq resolved

pal-e-services#33 is merged. CNPG cluster manifest is back under source control. Adding basketball database is now a SQL operation on the running cluster (per Constraints section — bootstrap.initdb only runs on creation).

Fix 1: CNPG database creation mechanism

Adding a database to an existing CNPG cluster is SQL, not manifest:

CREATE DATABASE basketball;
CREATE USER basketball WITH PASSWORD '...';
GRANT ALL PRIVILEGES ON DATABASE basketball TO basketball;

Run via: kubectl exec -n postgres pal-e-postgres-1 -- psql -U postgres -c "..."

Fix 2: Network policy — added to File Targets + AC

File: terraform/network-policies.tf (lines 175-179). Must add basketball-api namespace to the postgres ingress allow list.

Fix 3: File target paths corrected

• ~/pal-e-deployments/basketball-api/ → ~/pal-e-deployments/overlays/basketball-api/prod/
• ~/basketball-api/k8s/deployment.yaml → not the deploy target. ArgoCD image updater handles image tags. Kustomize overlay has env vars.
• ~/basketball-api/src/basketball_api/config.py — verify DATABASE_URL format works with new host

Fix 4: Backup verification AC reworded

cnpg-backup-verify CronJob checks WAL object freshness in MinIO by prefix, not per-database restore. AC reworded to: "Next daily Barman backup completes successfully after migration."

Updated File Targets (final)

1. terraform/network-policies.tf — add basketball-api to postgres namespace ingress allow list
2. ~/pal-e-deployments/overlays/basketball-api/prod/kustomization.yaml — update DATABASE_HOST env var to pal-e-postgres-rw.postgres.svc.cluster.local
3. ~/pal-e-deployments/overlays/basketball-api/prod/deployment-patch.yaml — update DATABASE env vars
4. ~/pal-e-deployments/overlays/basketball-api/prod/postgres.yaml — REMOVE (standalone postgres Deployment)
5. ~/pal-e-deployments/overlays/basketball-api/prod/pvc.yaml — REMOVE after verification (standalone PVC)

Updated Acceptance Criteria (final)

• basketball database + user created on CNPG cluster via SQL
• Network policy allows basketball-api namespace → postgres namespace
• basketball-api deployment points at pal-e-postgres-rw.postgres.svc.cluster.local
• pg_dump from standalone → pg_restore into CNPG verified (row counts match)
• App health check passes: curl basketball-api.tail5b443a.ts.net/health returns 200
• Next daily Barman backup completes successfully
• Old standalone postgres Deployment removed from kustomize overlay
• Old PVC retained for 7 days as rollback safety net, then deleted

Migration Steps (updated)

1. Add network policy rule (terraform apply)
2. Create database + user on CNPG via kubectl exec psql
3. pg_dump from standalone → pg_restore into CNPG (9MB, seconds)
4. Verify row counts: players, orders, registrations, teams, parents
5. Update kustomize overlay: new DATABASE_HOST, remove postgres.yaml
6. ArgoCD sync — basketball-api reconnects to CNPG
7. Verify health endpoint + spot-check data
8. Wait for next daily backup, confirm it includes basketball data
9. Remove standalone postgres Deployment from overlay (keep PVC 7 days)

## Refinement v2 (post review-417-2026-03-26 + CNPG manifest landed) ### Prereq resolved `pal-e-services#33` is merged. CNPG cluster manifest is back under source control. Adding `basketball` database is now a SQL operation on the running cluster (per Constraints section — `bootstrap.initdb` only runs on creation). ### Fix 1: CNPG database creation mechanism Adding a database to an existing CNPG cluster is SQL, not manifest: ```sql CREATE DATABASE basketball; CREATE USER basketball WITH PASSWORD '...'; GRANT ALL PRIVILEGES ON DATABASE basketball TO basketball; ``` Run via: `kubectl exec -n postgres pal-e-postgres-1 -- psql -U postgres -c "..."` ### Fix 2: Network policy — added to File Targets + AC File: `terraform/network-policies.tf` (lines 175-179). Must add `basketball-api` namespace to the postgres ingress allow list. ### Fix 3: File target paths corrected - ~~`~/pal-e-deployments/basketball-api/`~~ → `~/pal-e-deployments/overlays/basketball-api/prod/` - ~~`~/basketball-api/k8s/deployment.yaml`~~ → not the deploy target. ArgoCD image updater handles image tags. Kustomize overlay has env vars. - `~/basketball-api/src/basketball_api/config.py` — verify DATABASE_URL format works with new host ### Fix 4: Backup verification AC reworded `cnpg-backup-verify` CronJob checks WAL object freshness in MinIO by prefix, not per-database restore. AC reworded to: "Next daily Barman backup completes successfully after migration." ### Updated File Targets (final) 1. `terraform/network-policies.tf` — add `basketball-api` to postgres namespace ingress allow list 2. `~/pal-e-deployments/overlays/basketball-api/prod/kustomization.yaml` — update `DATABASE_HOST` env var to `pal-e-postgres-rw.postgres.svc.cluster.local` 3. `~/pal-e-deployments/overlays/basketball-api/prod/deployment-patch.yaml` — update DATABASE env vars 4. `~/pal-e-deployments/overlays/basketball-api/prod/postgres.yaml` — REMOVE (standalone postgres Deployment) 5. `~/pal-e-deployments/overlays/basketball-api/prod/pvc.yaml` — REMOVE after verification (standalone PVC) ### Updated Acceptance Criteria (final) - [ ] `basketball` database + user created on CNPG cluster via SQL - [ ] Network policy allows `basketball-api` namespace → `postgres` namespace - [ ] basketball-api deployment points at `pal-e-postgres-rw.postgres.svc.cluster.local` - [ ] `pg_dump` from standalone → `pg_restore` into CNPG verified (row counts match) - [ ] App health check passes: `curl basketball-api.tail5b443a.ts.net/health` returns 200 - [ ] Next daily Barman backup completes successfully - [ ] Old standalone postgres Deployment removed from kustomize overlay - [ ] Old PVC retained for 7 days as rollback safety net, then deleted ### Migration Steps (updated) 1. Add network policy rule (terraform apply) 2. Create database + user on CNPG via kubectl exec psql 3. pg_dump from standalone → pg_restore into CNPG (9MB, seconds) 4. Verify row counts: `players`, `orders`, `registrations`, `teams`, `parents` 5. Update kustomize overlay: new DATABASE_HOST, remove postgres.yaml 6. ArgoCD sync — basketball-api reconnects to CNPG 7. Verify health endpoint + spot-check data 8. Wait for next daily backup, confirm it includes basketball data 9. Remove standalone postgres Deployment from overlay (keep PVC 7 days)

forgejo_admin commented 2026-03-27 07:50:33 +00:00

Author

Owner

Copy link

Scope Review: NEEDS_REFINEMENT

Review note: review-417-2026-03-26-v2

Re-review after refinement v2. All 4 original issues were addressed. One new file target error found:

• File target #5 (pvc.yaml) is WRONG — pvc.yaml contains the photo-uploads PVC, NOT the postgres PVC. The postgres PVC (postgres-data) is defined inside postgres.yaml. Removing pvc.yaml as the ticket instructs would delete photo uploads and cause data loss. Fix: remove file target #5 entirely — postgres.yaml removal (target #4) already handles the standalone postgres PVC.
• Advisory (non-blocking): env var is BASKETBALL_DATABASE_URL (full connection string), not a separate DATABASE_HOST var as refinement v2 states. Update wording to prevent agent confusion.

## Scope Review: NEEDS_REFINEMENT Review note: `review-417-2026-03-26-v2` Re-review after refinement v2. All 4 original issues were addressed. One new file target error found: - **File target #5 (`pvc.yaml`) is WRONG** — `pvc.yaml` contains the `photo-uploads` PVC, NOT the postgres PVC. The postgres PVC (`postgres-data`) is defined inside `postgres.yaml`. Removing `pvc.yaml` as the ticket instructs would delete photo uploads and cause data loss. Fix: remove file target #5 entirely — `postgres.yaml` removal (target #4) already handles the standalone postgres PVC. - **Advisory (non-blocking):** env var is `BASKETBALL_DATABASE_URL` (full connection string), not a separate `DATABASE_HOST` var as refinement v2 states. Update wording to prevent agent confusion.

forgejo_admin commented 2026-03-27 07:51:05 +00:00

Author

Owner

Copy link

Refinement v3 (post review-417-2026-03-26-v2)

Fix 1: CRITICAL — pvc.yaml is photo-uploads, NOT postgres

pvc.yaml contains the photo-uploads PVC for player photos. DO NOT DELETE. The postgres PVC (postgres-data) is defined inside postgres.yaml. Removing postgres.yaml handles both the standalone Deployment and its PVC reference.

Corrected File Targets:

1. terraform/network-policies.tf — add basketball-api to postgres namespace ingress allow list
2. ~/pal-e-deployments/overlays/basketball-api/prod/deployment-patch.yaml — update BASKETBALL_DATABASE_URL connection string to point at CNPG
3. ~/pal-e-deployments/overlays/basketball-api/prod/postgres.yaml — REMOVE (standalone postgres Deployment + PVC)
4. ~/pal-e-deployments/overlays/basketball-api/prod/kustomization.yaml — remove postgres.yaml from resources list
5. pvc.yaml — DO NOT TOUCH (contains photo-uploads PVC, not postgres)

Fix 2: Env var name correction

The actual env var is BASKETBALL_DATABASE_URL (full connection string), not separate DATABASE_HOST/DATABASE_NAME vars. The connection string in deployment-patch.yaml line 29-30 needs updating:

postgresql://basketball:{password}@pal-e-postgres-rw.postgres.svc.cluster.local:5432/basketball

## Refinement v3 (post review-417-2026-03-26-v2) ### Fix 1: CRITICAL — pvc.yaml is photo-uploads, NOT postgres `pvc.yaml` contains the `photo-uploads` PVC for player photos. **DO NOT DELETE.** The postgres PVC (`postgres-data`) is defined inside `postgres.yaml`. Removing `postgres.yaml` handles both the standalone Deployment and its PVC reference. **Corrected File Targets:** 1. `terraform/network-policies.tf` — add `basketball-api` to postgres namespace ingress allow list 2. `~/pal-e-deployments/overlays/basketball-api/prod/deployment-patch.yaml` — update `BASKETBALL_DATABASE_URL` connection string to point at CNPG 3. `~/pal-e-deployments/overlays/basketball-api/prod/postgres.yaml` — REMOVE (standalone postgres Deployment + PVC) 4. `~/pal-e-deployments/overlays/basketball-api/prod/kustomization.yaml` — remove `postgres.yaml` from resources list 5. ~~`pvc.yaml`~~ — **DO NOT TOUCH** (contains photo-uploads PVC, not postgres) ### Fix 2: Env var name correction The actual env var is `BASKETBALL_DATABASE_URL` (full connection string), not separate `DATABASE_HOST`/`DATABASE_NAME` vars. The connection string in `deployment-patch.yaml` line 29-30 needs updating: ``` postgresql://basketball:{password}@pal-e-postgres-rw.postgres.svc.cluster.local:5432/basketball ```

forgejo_admin commented 2026-03-27 10:53:02 +00:00

Author

Owner

Copy link

Scope Review: READY

Review note: review-417-2026-03-26-v3

All 5 findings from previous reviews (v1: 4 issues, v2: 1 blocker + 1 advisory) verified as resolved in refinement v3. All file targets confirmed against codebase. Prereq (pal-e-services#33 CNPG manifest) is done. Traceability complete. Ticket is ready for execution.

Agent note: read refinement v3 comment for corrected file targets and acceptance criteria -- issue body still has original (stale) versions.

## Scope Review: READY Review note: `review-417-2026-03-26-v3` All 5 findings from previous reviews (v1: 4 issues, v2: 1 blocker + 1 advisory) verified as resolved in refinement v3. All file targets confirmed against codebase. Prereq (pal-e-services#33 CNPG manifest) is done. Traceability complete. Ticket is ready for execution. Agent note: read refinement v3 comment for corrected file targets and acceptance criteria -- issue body still has original (stale) versions.

forgejo_admin referenced this issue from a commit 2026-03-28 00:15:56 +00:00

fix: allow basketball-api namespace ingress to postgres namespace

forgejo_admin referenced this issue from a pull request that will close it, 2026-03-28 00:18:29 +00:00

fix: allow basketball-api ingress to postgres namespace (#187) #212

forgejo_admin referenced this issue 2026-03-28 17:42:32 +00:00

fix: allow basketball-api ingress to postgres namespace (#187) #212

forgejo_admin added the

status:approved

label 2026-03-28 17:42:33 +00:00

forgejo_admin closed this issue 2026-03-28 18:05:49 +00:00

forgejo_admin referenced this issue from a commit 2026-03-28 18:05:49 +00:00

fix: allow basketball-api ingress to postgres namespace (#187) (#212)

Sign in to join this conversation.

No Branch/Tag specified

Branches Tags

main

188-cross-repo-worktree-isolation-for-parall

111-fix-keycloak-probe

86-fix-rotate-woodpecker-api-token-in-salt

71-feat-deploy-pyrra-slo-manager-with-servi

64-hotfix-woodpecker-oauth-login-broken-for

18-fix-grafana-duplicate-default-datasource-v2

18-fix-grafana-duplicate-default-datasource

13-fix-cnpg-all-parameters

No results found.

Labels

Clear labels   

domain:backend

  

domain:devops

  

domain:frontend

  

status:approved

QA passed, awaiting merge approval

  

status:in-progress

Dev agent is actively working

  

status:needs-fix

QA found issues, back to dev

  

status:qa

PR submitted, awaiting QA review

  

type:bug

Bug fix

  

type:devops

Infrastructure/CI/config work

  

type:feature

New functionality

No labels

domain:backend

domain:devops

domain:frontend

status:approved

status:in-progress

status:needs-fix

status:qa

type:bug

type:devops

type:feature

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

forgejo_admin/pal-e-platform#187

No description provided.