Migrate Woodpecker CI from SQLite to Postgres (CNPG) #28

New issue

Closed

opened 2026-03-09 22:34:23 +00:00 by forgejo_admin · 1 comment

forgejo_admin commented

2026-03-09 22:34:23 +00:00

Owner

Lineage

plan-2026-02-26-tf-modularize-postgres → new phase

Repo

forgejo_admin/pal-e-platform

User Story

As a platform operator
I want Woodpecker CI to use Postgres instead of SQLite
So that CI logs are reliably retrievable and we eliminate SQLite from the stack

Context

Woodpecker uses SQLite by default. The K8s backend has a known log streaming bug (upstream #4409) where queue.Done: cannot ack workflow and stream: not found cause ALL pipeline logs to be empty at the API layer. Logs ARE stored in SQLite but can't be retrieved via API/UI. Affects all repos platform-wide. SQLite should not be on our stack — we have CNPG.

File Targets

Files the agent should modify:

terraform/main.tf -- add WOODPECKER_DATABASE_DRIVER and WOODPECKER_DATABASE_DATASOURCE env vars to server config

Files the agent should NOT touch:

Any non-Woodpecker terraform resources

Acceptance Criteria

Woodpecker server uses Postgres via CNPG cluster
Pipeline logs are retrievable via Woodpecker API/UI
tofu plan shows only Woodpecker helm_release changes
Existing pipeline history migrated or acceptable to lose

Test Expectations

Trigger a pipeline after migration and verify logs appear in UI
Verify queue.Done: cannot ack workflow errors are gone from server logs
Run command: kubectl logs woodpecker-server-0 -n woodpecker --tail=20

Constraints

Create woodpecker database in existing CNPG cluster
Use -target=helm_release.woodpecker for isolated apply
Pipeline history loss is acceptable if migration is complex

Checklist

PR opened
Tests pass
No unrelated changes

plan-2026-02-26-tf-modularize-postgres -- shared Postgres plan

### Lineage `plan-2026-02-26-tf-modularize-postgres` → new phase ### Repo `forgejo_admin/pal-e-platform` ### User Story As a platform operator I want Woodpecker CI to use Postgres instead of SQLite So that CI logs are reliably retrievable and we eliminate SQLite from the stack ### Context Woodpecker uses SQLite by default. The K8s backend has a known log streaming bug (upstream #4409) where `queue.Done: cannot ack workflow` and `stream: not found` cause ALL pipeline logs to be empty at the API layer. Logs ARE stored in SQLite but can't be retrieved via API/UI. Affects all repos platform-wide. SQLite should not be on our stack — we have CNPG. ### File Targets Files the agent should modify: - `terraform/main.tf` -- add WOODPECKER_DATABASE_DRIVER and WOODPECKER_DATABASE_DATASOURCE env vars to server config Files the agent should NOT touch: - Any non-Woodpecker terraform resources ### Acceptance Criteria - [ ] Woodpecker server uses Postgres via CNPG cluster - [ ] Pipeline logs are retrievable via Woodpecker API/UI - [ ] `tofu plan` shows only Woodpecker helm_release changes - [ ] Existing pipeline history migrated or acceptable to lose ### Test Expectations - [ ] Trigger a pipeline after migration and verify logs appear in UI - [ ] Verify `queue.Done: cannot ack workflow` errors are gone from server logs - Run command: `kubectl logs woodpecker-server-0 -n woodpecker --tail=20` ### Constraints - Create `woodpecker` database in existing CNPG cluster - Use `-target=helm_release.woodpecker` for isolated apply - Pipeline history loss is acceptable if migration is complex ### Checklist - [ ] PR opened - [ ] Tests pass - [ ] No unrelated changes ### Related - `plan-2026-02-26-tf-modularize-postgres` -- shared Postgres plan

forgejo_admin commented

2026-03-14 19:41:43 +00:00

Author

Owner

Updated approach (2026-03-14)

Original issue says "create woodpecker database in existing CNPG cluster." Updated plan creates a separate CNPG Cluster in woodpecker namespace for isolation:

Separate CNPG Cluster CR via kubernetes_manifest in terraform/main.tf
Own S3 creds secret in woodpecker namespace (reuses same MinIO IAM user)
DB credentials secret for connection string
Helm values: WOODPECKER_DATABASE_DRIVER=postgres + datasource
New woodpecker_db_password variable + CI secret
Pipeline history loss accepted (logs don't work anyway)

Rollback: remove DB env vars → Woodpecker falls back to SQLite PVC.

## Updated approach (2026-03-14) Original issue says "create woodpecker database in existing CNPG cluster." Updated plan creates a **separate CNPG Cluster** in `woodpecker` namespace for isolation: - Separate CNPG Cluster CR via `kubernetes_manifest` in `terraform/main.tf` - Own S3 creds secret in `woodpecker` namespace (reuses same MinIO IAM user) - DB credentials secret for connection string - Helm values: `WOODPECKER_DATABASE_DRIVER=postgres` + datasource - New `woodpecker_db_password` variable + CI secret - Pipeline history loss accepted (logs don't work anyway) **Rollback:** remove DB env vars → Woodpecker falls back to SQLite PVC.