Migrate all ArgoCD apps to pal-e-deployments centralized overlays #201

Closed
opened 2026-03-27 06:04:12 +00:00 by forgejo_admin · 3 comments

Type

Feature

Lineage

Decision made during incident #184 session. Lucas: "everyone should be using pal-e-deployments." Review (review-448-2026-03-26) revealed the Terraform migration is already done — only overlay creation + apply remain.

Repo

forgejo_admin/pal-e-deployments (overlay creation), forgejo_admin/pal-e-services (tofu apply)

User Story

As a platform operator
I want all application deployments managed through centralized kustomize overlays in pal-e-deployments
So that there's one deployment pattern, one place to update image tags, and the image updater can auto-deploy

Context

The Terraform migration is already done. All 9 services in pal-e-services/terraform/k3s.tfvars already have source_repo = "forgejo_admin/pal-e-deployments". ArgoCD Application resources are in pal-e-services/terraform/services.tf, NOT in pal-e-platform.

What's missing:

  1. pal-e-app overlay doesn't exist — it was intentionally deleted in pal-e-deployments commit 58efd8b. The tfvars already point to it, so tofu apply would break pal-e-app.
  2. tofu apply hasn't run in pal-e-services — the ArgoCD app source changes haven't been applied to the cluster.

All other overlays (basketball-api, gcal-scheduler, mcd-tracker, mcd-tracker-app, pal-e-docs, pal-e-mail, platform-validation, westsidekingsandqueens) already exist in pal-e-deployments.

File Targets

Files to create (in pal-e-deployments):

  • overlays/pal-e-app/prod/kustomization.yaml — kustomize overlay with image override
  • overlays/pal-e-app/prod/deployment-patch.yaml — env vars, port overrides
  • overlays/pal-e-app/prod/ingress.yaml — Tailscale funnel ingress
  • overlays/pal-e-app/prod/harbor-creds.enc.yaml — SOPS-encrypted Harbor pull secret

Files NOT to touch:

  • pal-e-platform/terraform/ — ArgoCD apps are NOT here
  • pal-e-services/terraform/k3s.tfvars — already configured correctly
  • pal-e-services/terraform/services.tf — already configured correctly
  • Other overlays — already exist and working

Acceptance Criteria

  • overlays/pal-e-app/prod/ overlay created in pal-e-deployments with proper kustomize image override
  • tofu apply in pal-e-services succeeds — all 9 ArgoCD apps point to pal-e-deployments
  • All 9 apps sync successfully from pal-e-deployments after apply
  • No app breaks during migration (especially pal-e-app)

Test Expectations

  • tofu plan -lock=false in pal-e-services shows ArgoCD app source changes
  • After apply: kubectl get applications -n argocd -o custom-columns='NAME:.metadata.name,REPO:.spec.source.repoURL,PATH:.spec.source.path' shows all apps on pal-e-deployments
  • All apps show Synced status
  • Run command: tofu plan -lock=false in pal-e-services/terraform/

Constraints

  • Create pal-e-app overlay BEFORE running tofu apply (tfvars already point to it)
  • Depends on #200 (internal URLs) — ArgoCD must be able to reach Forgejo reliably
  • tofu plan must include -lock=false
  • Reference existing overlays (e.g., westsidekingsandqueens) as template for pal-e-app

Checklist

  • PR opened (pal-e-deployments: pal-e-app overlay)
  • tofu apply run in pal-e-services
  • All apps verified synced
  • No unrelated changes
  • #200 — eliminate Tailscale hairpin (do first)
  • #184 (pal-e-platform) — parent incident
  • project-pal-e-platform
### Type Feature ### Lineage Decision made during incident #184 session. Lucas: "everyone should be using pal-e-deployments." Review (`review-448-2026-03-26`) revealed the Terraform migration is already done — only overlay creation + apply remain. ### Repo `forgejo_admin/pal-e-deployments` (overlay creation), `forgejo_admin/pal-e-services` (tofu apply) ### User Story As a platform operator I want all application deployments managed through centralized kustomize overlays in pal-e-deployments So that there's one deployment pattern, one place to update image tags, and the image updater can auto-deploy ### Context **The Terraform migration is already done.** All 9 services in `pal-e-services/terraform/k3s.tfvars` already have `source_repo = "forgejo_admin/pal-e-deployments"`. ArgoCD Application resources are in `pal-e-services/terraform/services.tf`, NOT in pal-e-platform. What's missing: 1. **pal-e-app overlay doesn't exist** — it was intentionally deleted in pal-e-deployments commit `58efd8b`. The tfvars already point to it, so `tofu apply` would break pal-e-app. 2. **`tofu apply` hasn't run** in pal-e-services — the ArgoCD app source changes haven't been applied to the cluster. All other overlays (basketball-api, gcal-scheduler, mcd-tracker, mcd-tracker-app, pal-e-docs, pal-e-mail, platform-validation, westsidekingsandqueens) already exist in pal-e-deployments. ### File Targets Files to create (in pal-e-deployments): - `overlays/pal-e-app/prod/kustomization.yaml` — kustomize overlay with image override - `overlays/pal-e-app/prod/deployment-patch.yaml` — env vars, port overrides - `overlays/pal-e-app/prod/ingress.yaml` — Tailscale funnel ingress - `overlays/pal-e-app/prod/harbor-creds.enc.yaml` — SOPS-encrypted Harbor pull secret Files NOT to touch: - `pal-e-platform/terraform/` — ArgoCD apps are NOT here - `pal-e-services/terraform/k3s.tfvars` — already configured correctly - `pal-e-services/terraform/services.tf` — already configured correctly - Other overlays — already exist and working ### Acceptance Criteria - [ ] `overlays/pal-e-app/prod/` overlay created in pal-e-deployments with proper kustomize image override - [ ] `tofu apply` in pal-e-services succeeds — all 9 ArgoCD apps point to pal-e-deployments - [ ] All 9 apps sync successfully from pal-e-deployments after apply - [ ] No app breaks during migration (especially pal-e-app) ### Test Expectations - [ ] `tofu plan -lock=false` in pal-e-services shows ArgoCD app source changes - [ ] After apply: `kubectl get applications -n argocd -o custom-columns='NAME:.metadata.name,REPO:.spec.source.repoURL,PATH:.spec.source.path'` shows all apps on pal-e-deployments - [ ] All apps show Synced status - Run command: `tofu plan -lock=false` in `pal-e-services/terraform/` ### Constraints - Create pal-e-app overlay BEFORE running tofu apply (tfvars already point to it) - Depends on #200 (internal URLs) — ArgoCD must be able to reach Forgejo reliably - `tofu plan` must include `-lock=false` - Reference existing overlays (e.g., westsidekingsandqueens) as template for pal-e-app ### Checklist - [ ] PR opened (pal-e-deployments: pal-e-app overlay) - [ ] tofu apply run in pal-e-services - [ ] All apps verified synced - [ ] No unrelated changes ### Related - #200 — eliminate Tailscale hairpin (do first) - #184 (pal-e-platform) — parent incident - `project-pal-e-platform`
Author
Owner

Scope Review: NEEDS_REFINEMENT

Review note: review-448-2026-03-26

The issue's premise is outdated -- most of the described work is already done.

Key findings:

  • k3s.tfvars already migrated: All 9 services in pal-e-services/terraform/k3s.tfvars already have source_repo = "forgejo_admin/pal-e-deployments". The Terraform config changes are complete.
  • Overlays already exist: gcal-scheduler and platform-validation overlays exist (issue says "No -- create"). Only pal-e-app overlay is truly missing (deleted in pal-e-deployments commit 58efd8b).
  • Wrong repo: Issue is filed on pal-e-platform but ArgoCD apps are defined in pal-e-services/terraform/services.tf. No changes needed in pal-e-platform.
  • Wrong file target: terraform/main.tf in pal-e-platform does not contain ArgoCD Application resources.
  • Live state drift: kubectl confirms only 3 of 9 apps currently point at pal-e-deployments. The remaining 6 will switch on next tofu apply in pal-e-services.
  • Blast radius warning: pal-e-app overlay must be created in pal-e-deployments BEFORE apply, or pal-e-app will break.

Actual remaining work: (1) create pal-e-app overlay in pal-e-deployments, (2) run tofu apply in pal-e-services.

## Scope Review: NEEDS_REFINEMENT Review note: `review-448-2026-03-26` **The issue's premise is outdated -- most of the described work is already done.** Key findings: - **k3s.tfvars already migrated**: All 9 services in `pal-e-services/terraform/k3s.tfvars` already have `source_repo = "forgejo_admin/pal-e-deployments"`. The Terraform config changes are complete. - **Overlays already exist**: gcal-scheduler and platform-validation overlays exist (issue says "No -- create"). Only `pal-e-app` overlay is truly missing (deleted in pal-e-deployments commit 58efd8b). - **Wrong repo**: Issue is filed on pal-e-platform but ArgoCD apps are defined in `pal-e-services/terraform/services.tf`. No changes needed in pal-e-platform. - **Wrong file target**: `terraform/main.tf` in pal-e-platform does not contain ArgoCD Application resources. - **Live state drift**: kubectl confirms only 3 of 9 apps currently point at pal-e-deployments. The remaining 6 will switch on next `tofu apply` in pal-e-services. - **Blast radius warning**: pal-e-app overlay must be created in pal-e-deployments BEFORE apply, or pal-e-app will break. Actual remaining work: (1) create pal-e-app overlay in pal-e-deployments, (2) run `tofu apply` in pal-e-services.
Author
Owner

Scope Re-Review: READY

Review note: review-448-2026-03-26 (updated from NEEDS_REFINEMENT to READY)

All 6 findings from the initial review have been addressed in the rewritten issue body:

  1. Context -- accurately states Terraform migration already done, only overlay + apply remain
  2. File Targets -- correctly scoped to pal-e-app overlay (4 files to create), explicit "do NOT touch" list verified against codebase
  3. Repo field -- correctly identifies pal-e-deployments + pal-e-services (not pal-e-platform)
  4. Architecture table -- removed; replaced with accurate prose in Context
  5. #196 dependency -- correctly omitted (verified: pal-e-services has no MinIO provider, so #196 is not a blocker)
  6. Scope -- appropriately reduced to 1 overlay creation + 1 tofu apply

Verified against codebase:

  • overlays/pal-e-app/ does NOT exist (confirmed deleted in commit 58efd8b)
  • All 9 services in k3s.tfvars already have source_repo = "forgejo_admin/pal-e-deployments"
  • argocd_application resource is in pal-e-services/terraform/services.tf, NOT in pal-e-platform
  • All 8 other overlays exist with matching file structure (kustomization.yaml, deployment-patch.yaml, ingress.yaml, harbor-creds.enc.yaml)
  • Dependency #200 (hairpin fix) is still open, correctly documented as prerequisite

VERDICT: READY -- ticket can move to next_up when #200 is resolved.

## Scope Re-Review: READY Review note: `review-448-2026-03-26` (updated from NEEDS_REFINEMENT to READY) All 6 findings from the initial review have been addressed in the rewritten issue body: 1. **Context** -- accurately states Terraform migration already done, only overlay + apply remain 2. **File Targets** -- correctly scoped to pal-e-app overlay (4 files to create), explicit "do NOT touch" list verified against codebase 3. **Repo field** -- correctly identifies pal-e-deployments + pal-e-services (not pal-e-platform) 4. **Architecture table** -- removed; replaced with accurate prose in Context 5. **#196 dependency** -- correctly omitted (verified: pal-e-services has no MinIO provider, so #196 is not a blocker) 6. **Scope** -- appropriately reduced to 1 overlay creation + 1 tofu apply **Verified against codebase:** - `overlays/pal-e-app/` does NOT exist (confirmed deleted in commit `58efd8b`) - All 9 services in `k3s.tfvars` already have `source_repo = "forgejo_admin/pal-e-deployments"` - `argocd_application` resource is in `pal-e-services/terraform/services.tf`, NOT in pal-e-platform - All 8 other overlays exist with matching file structure (kustomization.yaml, deployment-patch.yaml, ingress.yaml, harbor-creds.enc.yaml) - Dependency #200 (hairpin fix) is still open, correctly documented as prerequisite **VERDICT: READY** -- ticket can move to next_up when #200 is resolved.
Author
Owner

Scope Review: READY (post-completion confirmation)

Review note: review-448-2026-03-27
All work verified complete. Overlay files exist, all 9 tfvars entries point to pal-e-deployments, dependency #200 resolved, issue closed.

  • All 4 file targets verified present in pal-e-deployments/overlays/pal-e-app/prod/
  • "Do NOT touch" list respected -- no pal-e-platform files modified
  • Discovered scope tracked: board item #464 (wire update-kustomize-tag into all 9 repos) in backlog
## Scope Review: READY (post-completion confirmation) Review note: `review-448-2026-03-27` All work verified complete. Overlay files exist, all 9 tfvars entries point to pal-e-deployments, dependency #200 resolved, issue closed. - All 4 file targets verified present in `pal-e-deployments/overlays/pal-e-app/prod/` - "Do NOT touch" list respected -- no pal-e-platform files modified - Discovered scope tracked: board item #464 (wire update-kustomize-tag into all 9 repos) in backlog
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
forgejo_admin/pal-e-platform#201
No description provided.