forgejo_admin/pal-e-platform

Fork 0

fix: wire all Woodpecker secrets through terraform helm values #84

New issue

Closed

opened 2026-03-16 00:31:28 +00:00 by forgejo_admin · 1 comment

forgejo_admin commented

2026-03-16 00:31:28 +00:00

Owner

Lineage

plan-pal-e-platform → Phase 17a (Woodpecker Secrets Hardening)

Repo

forgejo_admin/pal-e-platform

User Story

As a platform operator
I want all Woodpecker secrets permanently wired through terraform
So that tofu apply never breaks Woodpecker, and DB migrations don't invalidate API tokens

Context

Every Woodpecker CNPG DB migration breaks 4 secrets across 5 consumers. This has happened TWICE. Root cause: Woodpecker generates a random jwt-secret in server_configs table on every fresh DB. The helm values don't wire secrets from terraform variables — DB password is empty in the connection string, agent secret has no value, and there's no persistent encryption key.

Current k3s.tfvars already has all the values:

woodpecker_api_token = "eyJ..."
woodpecker_db_password = "kM3L4AhLNiuMhIY7tMQ"
woodpecker_agent_secret = "3e053aaa..."

But the helm template doesn't use them properly.

File Targets

Files the agent should modify:

terraform/main.tf — Woodpecker helm_release resource:
1. Add WOODPECKER_ENCRYPTION_KEY env var to server (new terraform variable woodpecker_encryption_key, 32-byte hex)
2. Interpolate woodpecker_db_password into WOODPECKER_DATABASE_DATASOURCE URL: change postgres://woodpecker:@ to postgres://woodpecker:${var.woodpecker_db_password}@
3. Set WOODPECKER_AGENT_SECRET env var on agent to var.woodpecker_agent_secret (currently empty)
terraform/variables.tf — Add woodpecker_encryption_key variable (sensitive string)
Makefile — Add woodpecker_db_password, woodpecker_agent_secret, woodpecker_encryption_key to TF_SECRET_VARS

Files the agent should NOT touch:

k3s.tfvars — already has the values (except woodpecker_encryption_key which Lucas will add manually)
Salt pillar — separate step, not in this PR
.woodpecker.yaml — CI pipeline doesn't need these secrets

Acceptance Criteria

WOODPECKER_DATABASE_DATASOURCE contains ${var.woodpecker_db_password} in the connection string
WOODPECKER_AGENT_SECRET env var on agent is set to var.woodpecker_agent_secret
WOODPECKER_ENCRYPTION_KEY env var on server is set to var.woodpecker_encryption_key
woodpecker_encryption_key declared in variables.tf as sensitive
Makefile TF_SECRET_VARS includes all 3 new entries
tofu validate passes
tofu fmt produces no changes
tofu plan -lock=false shows only Woodpecker helm release changes (use placeholder values for missing vars)

Test Expectations

tofu validate — must pass
tofu plan -lock=false — review for expected Woodpecker changes only
grep the rendered helm values for empty passwords (should find none)

Constraints

Must run tofu fmt before committing
Must run tofu validate before committing
Do NOT run tofu apply
Include tofu plan -lock=false output in PR description
The woodpecker_encryption_key variable won't have a value yet in k3s.tfvars — use a default empty string with a validation block or pass -var="woodpecker_encryption_key=placeholder" for plan

Checklist

PR opened
tofu validate passes
No unrelated changes

phase-platform-17a-woodpecker-secrets — parent phase
todo-woodpecker-secrets-terraform — the TODO this resolves
pal-e-platform — project

### Lineage `plan-pal-e-platform` → Phase 17a (Woodpecker Secrets Hardening) ### Repo `forgejo_admin/pal-e-platform` ### User Story As a platform operator I want all Woodpecker secrets permanently wired through terraform So that `tofu apply` never breaks Woodpecker, and DB migrations don't invalidate API tokens ### Context Every Woodpecker CNPG DB migration breaks 4 secrets across 5 consumers. This has happened TWICE. Root cause: Woodpecker generates a random `jwt-secret` in `server_configs` table on every fresh DB. The helm values don't wire secrets from terraform variables — DB password is empty in the connection string, agent secret has no value, and there's no persistent encryption key. Current `k3s.tfvars` already has all the values: ``` woodpecker_api_token = "eyJ..." woodpecker_db_password = "kM3L4AhLNiuMhIY7tMQ" woodpecker_agent_secret = "3e053aaa..." ``` But the helm template doesn't use them properly. ### File Targets Files the agent should modify: - `terraform/main.tf` — Woodpecker helm_release resource: 1. Add `WOODPECKER_ENCRYPTION_KEY` env var to server (new terraform variable `woodpecker_encryption_key`, 32-byte hex) 2. Interpolate `woodpecker_db_password` into `WOODPECKER_DATABASE_DATASOURCE` URL: change `postgres://woodpecker:@` to `postgres://woodpecker:${var.woodpecker_db_password}@` 3. Set `WOODPECKER_AGENT_SECRET` env var on agent to `var.woodpecker_agent_secret` (currently empty) - `terraform/variables.tf` — Add `woodpecker_encryption_key` variable (sensitive string) - `Makefile` — Add `woodpecker_db_password`, `woodpecker_agent_secret`, `woodpecker_encryption_key` to `TF_SECRET_VARS` Files the agent should NOT touch: - `k3s.tfvars` — already has the values (except `woodpecker_encryption_key` which Lucas will add manually) - Salt pillar — separate step, not in this PR - `.woodpecker.yaml` — CI pipeline doesn't need these secrets ### Acceptance Criteria - [ ] `WOODPECKER_DATABASE_DATASOURCE` contains `${var.woodpecker_db_password}` in the connection string - [ ] `WOODPECKER_AGENT_SECRET` env var on agent is set to `var.woodpecker_agent_secret` - [ ] `WOODPECKER_ENCRYPTION_KEY` env var on server is set to `var.woodpecker_encryption_key` - [ ] `woodpecker_encryption_key` declared in `variables.tf` as sensitive - [ ] Makefile `TF_SECRET_VARS` includes all 3 new entries - [ ] `tofu validate` passes - [ ] `tofu fmt` produces no changes - [ ] `tofu plan -lock=false` shows only Woodpecker helm release changes (use placeholder values for missing vars) ### Test Expectations - [ ] `tofu validate` — must pass - [ ] `tofu plan -lock=false` — review for expected Woodpecker changes only - [ ] grep the rendered helm values for empty passwords (should find none) ### Constraints - Must run `tofu fmt` before committing - Must run `tofu validate` before committing - Do NOT run `tofu apply` - Include `tofu plan -lock=false` output in PR description - The `woodpecker_encryption_key` variable won't have a value yet in `k3s.tfvars` — use a default empty string with a validation block or pass `-var="woodpecker_encryption_key=placeholder"` for plan ### Checklist - [ ] PR opened - [ ] `tofu validate` passes - [ ] No unrelated changes ### Related - `phase-platform-17a-woodpecker-secrets` — parent phase - `todo-woodpecker-secrets-terraform` — the TODO this resolves - `pal-e-platform` — project

forgejo_admin added the

status:in-progress

type:devops

labels

2026-03-16 00:31:35 +00:00

forgejo_admin referenced this issue from a commit

2026-03-16 00:33:45 +00:00

fix: wire all Woodpecker secrets through terraform helm values

forgejo_admin referenced this issue from a pull request that will close it,

2026-03-16 00:34:11 +00:00

fix: wire all Woodpecker secrets through terraform helm values #85

forgejo_admin added the

status:qa

label

2026-03-16 00:35:52 +00:00

forgejo_admin commented

2026-03-16 00:38:16 +00:00

Author

Owner

PR #85 Review

DOMAIN REVIEW

Tech stack: Terraform/Helm (OpenTofu). Three files changed: Makefile, terraform/main.tf, terraform/variables.tf. 14 additions, 1 deletion.

Terraform style:

set_sensitive block for WOODPECKER_ENCRYPTION_KEY is correctly placed at server.env.WOODPECKER_ENCRYPTION_KEY within the helm_release.woodpecker resource (line ~767-771). It sits between the Forgejo OAuth secrets and the existing WOODPECKER_AGENT_SECRET set_sensitive block. Correct Helm value path (server.env.*), correct type = "string".
Variable declaration in variables.tf follows the established pattern: sensitive = true, descriptive description field, type = string. Placed after woodpecker_agent_secret -- logical grouping.
Makefile TF_SECRET_VARS addition: the three new entries (woodpecker_db_password, woodpecker_agent_secret, woodpecker_encryption_key) are appended as a single new line with proper backslash continuation on the previous line. No duplicates -- verified woodpecker_forgejo_client and woodpecker_forgejo_secret were already there, and none of the three new entries existed previously.

Pre-existing patterns confirmed (not part of this diff):

WOODPECKER_DATABASE_DATASOURCE on line 718 interpolates var.woodpecker_db_password directly in the values block (not set_sensitive). This means the password appears in plan output and state. Pre-existing -- not introduced here, and fixing it would be scope creep for this issue.
WOODPECKER_AGENT_SECRET already has set_sensitive on both server.env (line 774) and agent.env (line 780). Correct.

BLOCKERS

None.

No secrets or credentials in code.
No new functionality requiring test coverage (pure infrastructure wiring).
No unvalidated user input.
No DRY violations -- each secret is wired exactly once.

NITS

Salt pillar gaps (pre-existing, out of scope): woodpecker_db_password and woodpecker_encryption_key are not in salt/pillar/secrets/platform.sls (GPG-encrypted values) or salt/pillar/secrets_registry.sls (metadata). Only woodpecker_agent_secret has both. The PR acknowledges this as an operator step in the Review Checklist ("Salt pillar updated with woodpecker_encryption_key"). This means make tofu-secrets will silently skip these two variables until pillar is updated -- Terraform will then fail on missing required variables. The operator is warned.
k3s.tfvars.example incomplete (pre-existing): The example file has woodpecker_db_password but is missing woodpecker_agent_secret and woodpecker_encryption_key. Not part of this diff, but worth tracking as a follow-up nit.
WOODPECKER_DATABASE_DATASOURCE not using set_sensitive (pre-existing): The DB password is interpolated in the values yamlencode block (line 718) rather than via set_sensitive. This leaks the password in tofu plan output and state. Consider extracting to set_sensitive in a future PR. Not in scope here.

SOP COMPLIANCE

Branch named after issue: 84-fix-wire-all-woodpecker-secrets-through (starts with 84)
PR body follows template: Summary, Changes, Test Plan, Review Checklist, Related sections all present
Related references plan slug: plan-pal-e-platform Phase 17a
Closes #84 present in PR body
No secrets committed -- all values are variable references, no plaintext credentials
No unrelated file changes -- all 3 files are directly relevant to the issue
tofu fmt and tofu validate confirmed passing per PR body
tofu plan output not included -- acknowledged in PR body as requiring server-side execution with real state. Operator step.

PROCESS OBSERVATIONS

Change failure risk: LOW. This is additive wiring -- new set_sensitive block and Makefile entries. No destructive changes. Existing Helm release will get one new env var (WOODPECKER_ENCRYPTION_KEY). The Woodpecker pod will restart on next tofu apply, which is expected.
Operational dependency: The operator must (1) add woodpecker_encryption_key to Salt pillar with a GPG-encrypted value, (2) add registry entries for both woodpecker_db_password and woodpecker_encryption_key to secrets_registry.sls, and (3) run make tofu-secrets before tofu apply. The PR Review Checklist correctly surfaces these as unchecked operator steps.
Pre-existing debt: The datasource URL password leak (nit #3) should get its own issue. It predates this PR but is the kind of thing that causes audit findings.

VERDICT: APPROVED

Clean, minimal, correctly scoped fix. All three secrets properly wired. No blockers. Three pre-existing nits noted for follow-up.

## PR #85 Review ### DOMAIN REVIEW **Tech stack:** Terraform/Helm (OpenTofu). Three files changed: `Makefile`, `terraform/main.tf`, `terraform/variables.tf`. 14 additions, 1 deletion. **Terraform style:** - `set_sensitive` block for `WOODPECKER_ENCRYPTION_KEY` is correctly placed at `server.env.WOODPECKER_ENCRYPTION_KEY` within the `helm_release.woodpecker` resource (line ~767-771). It sits between the Forgejo OAuth secrets and the existing `WOODPECKER_AGENT_SECRET` `set_sensitive` block. Correct Helm value path (`server.env.*`), correct `type = "string"`. - Variable declaration in `variables.tf` follows the established pattern: `sensitive = true`, descriptive `description` field, `type = string`. Placed after `woodpecker_agent_secret` -- logical grouping. - Makefile `TF_SECRET_VARS` addition: the three new entries (`woodpecker_db_password`, `woodpecker_agent_secret`, `woodpecker_encryption_key`) are appended as a single new line with proper backslash continuation on the previous line. No duplicates -- verified `woodpecker_forgejo_client` and `woodpecker_forgejo_secret` were already there, and none of the three new entries existed previously. **Pre-existing patterns confirmed (not part of this diff):** - `WOODPECKER_DATABASE_DATASOURCE` on line 718 interpolates `var.woodpecker_db_password` directly in the `values` block (not `set_sensitive`). This means the password appears in plan output and state. Pre-existing -- not introduced here, and fixing it would be scope creep for this issue. - `WOODPECKER_AGENT_SECRET` already has `set_sensitive` on both `server.env` (line 774) and `agent.env` (line 780). Correct. ### BLOCKERS None. - No secrets or credentials in code. - No new functionality requiring test coverage (pure infrastructure wiring). - No unvalidated user input. - No DRY violations -- each secret is wired exactly once. ### NITS 1. **Salt pillar gaps (pre-existing, out of scope):** `woodpecker_db_password` and `woodpecker_encryption_key` are not in `salt/pillar/secrets/platform.sls` (GPG-encrypted values) or `salt/pillar/secrets_registry.sls` (metadata). Only `woodpecker_agent_secret` has both. The PR acknowledges this as an operator step in the Review Checklist ("Salt pillar updated with `woodpecker_encryption_key`"). This means `make tofu-secrets` will silently skip these two variables until pillar is updated -- Terraform will then fail on missing required variables. The operator is warned. 2. **`k3s.tfvars.example` incomplete (pre-existing):** The example file has `woodpecker_db_password` but is missing `woodpecker_agent_secret` and `woodpecker_encryption_key`. Not part of this diff, but worth tracking as a follow-up nit. 3. **`WOODPECKER_DATABASE_DATASOURCE` not using `set_sensitive` (pre-existing):** The DB password is interpolated in the `values` yamlencode block (line 718) rather than via `set_sensitive`. This leaks the password in `tofu plan` output and state. Consider extracting to `set_sensitive` in a future PR. Not in scope here. ### SOP COMPLIANCE - [x] Branch named after issue: `84-fix-wire-all-woodpecker-secrets-through` (starts with `84`) - [x] PR body follows template: Summary, Changes, Test Plan, Review Checklist, Related sections all present - [x] Related references plan slug: `plan-pal-e-platform` Phase 17a - [x] `Closes #84` present in PR body - [x] No secrets committed -- all values are variable references, no plaintext credentials - [x] No unrelated file changes -- all 3 files are directly relevant to the issue - [x] `tofu fmt` and `tofu validate` confirmed passing per PR body - [ ] `tofu plan` output not included -- acknowledged in PR body as requiring server-side execution with real state. Operator step. ### PROCESS OBSERVATIONS - **Change failure risk: LOW.** This is additive wiring -- new `set_sensitive` block and Makefile entries. No destructive changes. Existing Helm release will get one new env var (`WOODPECKER_ENCRYPTION_KEY`). The Woodpecker pod will restart on next `tofu apply`, which is expected. - **Operational dependency:** The operator must (1) add `woodpecker_encryption_key` to Salt pillar with a GPG-encrypted value, (2) add registry entries for both `woodpecker_db_password` and `woodpecker_encryption_key` to `secrets_registry.sls`, and (3) run `make tofu-secrets` before `tofu apply`. The PR Review Checklist correctly surfaces these as unchecked operator steps. - **Pre-existing debt:** The datasource URL password leak (nit #3) should get its own issue. It predates this PR but is the kind of thing that causes audit findings. ### VERDICT: APPROVED Clean, minimal, correctly scoped fix. All three secrets properly wired. No blockers. Three pre-existing nits noted for follow-up.

forgejo_admin added the

status:approved

label

2026-03-16 00:38:21 +00:00

forgejo_admin referenced this issue from a commit

2026-03-16 00:52:07 +00:00

fix: add woodpecker_db_password + agent_secret to Makefile TF_SECRET_VARS

forgejo_admin closed this issue

2026-03-16 01:22:28 +00:00