Fix deployment image tag to correct merge commit SHA #92

New issue

Closed

opened 2026-03-06 22:22:46 +00:00 by forgejo_admin · 0 comments

forgejo_admin commented

2026-03-06 22:22:46 +00:00

Contributor

Plan

plan-2026-02-26-tf-modularize-postgres -- Phase 5 (hotfix during deployment)

Repo

forgejo_admin/pal-e-docs

User Story

As a platform operator
I want the deployment.yaml image tag to match an actual Harbor image tag
So that ArgoCD can sync without causing an outage

Context

PR #88 pinned k8s/deployment.yaml to c85a39da... — a squash commit SHA from PR #86's branch. Woodpecker CI tags images with ${CI_COMMIT_SHA}, which is the merge commit on main, not the branch commit. That tag never existed in Harbor.

The bug was masked by the ArgoCD Image Updater ghost override (.argocd-source-pal-e-docs.yaml), which was silently forcing a different image. When PR #91 removed the override, ArgoCD tried to pull the non-existent tag and the pod entered ImagePullBackOff.

Currently running via manual kubectl patch with ArgoCD auto-sync disabled. Git and cluster are out of sync.

Full incident report: incident-phase5-deployment-outage-2026-03-06 in pal-e-docs knowledge base.

File Targets

Files the agent should modify:

k8s/deployment.yaml -- update image tag to 2eddd7660bccbd38f5cc2e53907f2f5485a74ef0 (latest merge commit on main, confirmed in Harbor)

Files the agent should NOT touch:

Everything else -- this is a single-line fix

Acceptance Criteria

When ArgoCD syncs, then it can pull the image without error
When I check kubectl get pods -n pal-e-docs, then the pod is Running with the correct image

Test Expectations

CI passes (test + build-and-push)
No additional tests needed -- this is a manifest change

Constraints

This PR's own merge commit will trigger a new CI build with a new SHA. That's expected — the next image tag update will use the new merge SHA. This PR just fixes the immediate broken state.
After merge: re-enable ArgoCD auto-sync (see post-merge steps below)

Post-merge steps (for Betty Sue, not the agent):

Wait for CI pipeline to complete
Re-enable ArgoCD auto-sync: kubectl patch application pal-e-docs -n argocd --type='json' -p='[{"op":"add","path":"/spec/syncPolicy/automated","value":{"prune":true,"selfHeal":true}}]'
Verify pod rolls to the correct image

Checklist

PR opened
Tests pass
No unrelated changes

pal-e-docs -- project
incident-phase5-deployment-outage-2026-03-06 -- incident report
concept-argocd-ghost-override -- root cause explanation
Closes #87

### Plan `plan-2026-02-26-tf-modularize-postgres` -- Phase 5 (hotfix during deployment) ### Repo `forgejo_admin/pal-e-docs` ### User Story As a platform operator I want the deployment.yaml image tag to match an actual Harbor image tag So that ArgoCD can sync without causing an outage ### Context PR #88 pinned `k8s/deployment.yaml` to `c85a39da...` — a squash commit SHA from PR #86's branch. Woodpecker CI tags images with `${CI_COMMIT_SHA}`, which is the **merge commit** on main, not the branch commit. That tag never existed in Harbor. The bug was masked by the ArgoCD Image Updater ghost override (`.argocd-source-pal-e-docs.yaml`), which was silently forcing a different image. When PR #91 removed the override, ArgoCD tried to pull the non-existent tag and the pod entered ImagePullBackOff. Currently running via manual kubectl patch with ArgoCD auto-sync disabled. Git and cluster are out of sync. Full incident report: `incident-phase5-deployment-outage-2026-03-06` in pal-e-docs knowledge base. ### File Targets Files the agent should modify: - `k8s/deployment.yaml` -- update image tag to `2eddd7660bccbd38f5cc2e53907f2f5485a74ef0` (latest merge commit on main, confirmed in Harbor) Files the agent should NOT touch: - Everything else -- this is a single-line fix ### Acceptance Criteria - [ ] When ArgoCD syncs, then it can pull the image without error - [ ] When I check `kubectl get pods -n pal-e-docs`, then the pod is Running with the correct image ### Test Expectations - [ ] CI passes (test + build-and-push) - No additional tests needed -- this is a manifest change ### Constraints - This PR's own merge commit will trigger a new CI build with a new SHA. That's expected — the next image tag update will use the new merge SHA. This PR just fixes the immediate broken state. - After merge: re-enable ArgoCD auto-sync (see post-merge steps below) Post-merge steps (for Betty Sue, not the agent): 1. Wait for CI pipeline to complete 2. Re-enable ArgoCD auto-sync: `kubectl patch application pal-e-docs -n argocd --type='json' -p='[{"op":"add","path":"/spec/syncPolicy/automated","value":{"prune":true,"selfHeal":true}}]'` 3. Verify pod rolls to the correct image ### Checklist - [ ] PR opened - [ ] Tests pass - [ ] No unrelated changes ### Related - `pal-e-docs` -- project - `incident-phase5-deployment-outage-2026-03-06` -- incident report - `concept-argocd-ghost-override` -- root cause explanation - Closes #87