Bug: Woodpecker webhook not firing on Forgejo squash merge — missed pipeline for basketball-api #382 #273

New issue

Open

opened 2026-04-07 18:27:53 +00:00 by forgejo_admin · 3 comments

forgejo_admin commented

2026-04-07 18:27:53 +00:00

Contributor

Type

Bug

Lineage

Standalone — discovered during CRM incident response 2026-04-07. CI test fix PR #382 merged on basketball-api but no Woodpecker pipeline triggered.

Repo

forgejo_admin/pal-e-platform

What Broke

When PR #382 was squash-merged on forgejo_admin/basketball-api, Forgejo did not fire a webhook to Woodpecker. No pipeline was created for the merge commit (801bc43). The Woodpecker queue showed 4 idle workers, 0 pending — the event simply never arrived.

This blocked deployment of two critical bug fixes (#377 photo placeholder, #378 teams 422) because the CI test fix needed to land first for update-kustomize-tag to run.

Blast radius: HIGH — Silent webhook failures on squash merge affect all 8+ repos with Woodpecker CI. Any squash merge could silently fail to trigger a pipeline.

Repro Steps

Squash-merge a PR on basketball-api
Check Woodpecker pipelines for a new push event
Observe: no pipeline created for the merge commit
Woodpecker queue shows idle workers, no pending jobs

Expected Behavior

Every squash merge to main triggers a Woodpecker push pipeline.

Environment

Cluster/namespace: prod / woodpecker + forgejo
Woodpecker queue: 4 workers, 0 pending at time of incident
Forgejo merge commit: 801bc436660ff1456d7150743193f3144a427c49

Acceptance Criteria

Root cause identified (Forgejo webhook config, Woodpecker webhook receiver, rate limiting, etc.)
Verify webhook delivery logs in Forgejo admin (Settings → Webhooks → Recent Deliveries)
Fix applied so squash merges reliably trigger pipelines
Verify fix across multiple repos (not just basketball-api)
Workaround documented if this is a known Forgejo/Woodpecker issue

pal-e-platform — CI infrastructure
forgejo_admin/basketball-api — affected repo
forgejo_admin/pal-e-platform #274 — companion bug (kustomize-tag skipped on test failure)
Manual kustomize tag update was required to deploy

### Type Bug ### Lineage Standalone — discovered during CRM incident response 2026-04-07. CI test fix PR #382 merged on basketball-api but no Woodpecker pipeline triggered. ### Repo `forgejo_admin/pal-e-platform` ### What Broke When PR #382 was squash-merged on `forgejo_admin/basketball-api`, Forgejo did not fire a webhook to Woodpecker. No pipeline was created for the merge commit (`801bc43`). The Woodpecker queue showed 4 idle workers, 0 pending — the event simply never arrived. This blocked deployment of two critical bug fixes (#377 photo placeholder, #378 teams 422) because the CI test fix needed to land first for `update-kustomize-tag` to run. **Blast radius: HIGH** — Silent webhook failures on squash merge affect all 8+ repos with Woodpecker CI. Any squash merge could silently fail to trigger a pipeline. ### Repro Steps 1. Squash-merge a PR on basketball-api 2. Check Woodpecker pipelines for a new push event 3. Observe: no pipeline created for the merge commit 4. Woodpecker queue shows idle workers, no pending jobs ### Expected Behavior Every squash merge to main triggers a Woodpecker push pipeline. ### Environment - Cluster/namespace: prod / woodpecker + forgejo - Woodpecker queue: 4 workers, 0 pending at time of incident - Forgejo merge commit: `801bc436660ff1456d7150743193f3144a427c49` ### Acceptance Criteria - [ ] Root cause identified (Forgejo webhook config, Woodpecker webhook receiver, rate limiting, etc.) - [ ] Verify webhook delivery logs in Forgejo admin (Settings → Webhooks → Recent Deliveries) - [ ] Fix applied so squash merges reliably trigger pipelines - [ ] Verify fix across multiple repos (not just basketball-api) - [ ] Workaround documented if this is a known Forgejo/Woodpecker issue ### Related - `pal-e-platform` — CI infrastructure - `forgejo_admin/basketball-api` — affected repo - `forgejo_admin/pal-e-platform #274` — companion bug (kustomize-tag skipped on test failure) - Manual kustomize tag update was required to deploy

forgejo_admin commented

2026-04-07 19:19:35 +00:00

Author

Contributor

Scope Review: NEEDS_REFINEMENT

Review note: review-273-2026-04-07

Well-scoped bug with clear symptoms and good repro steps. Six refinements needed before dispatch:

[LABEL] Replace story:PLAT-S2 with story:superuser-deploy on board item #881 -- PLAT-S2 is not a registered user story in project-pal-e-platform
[SCOPE] Create architecture note arch-woodpecker in pal-e-docs
[SCOPE] Create architecture note arch-forgejo in pal-e-docs
[BODY] Add cross-references to related issues #254, #259, #274 in the Related section
[BODY] Add AC: "Verify webhook config is consistent across all repos with Woodpecker pipelines"
[LABEL] Set board item #881 title (currently null)

## Scope Review: NEEDS_REFINEMENT Review note: `review-273-2026-04-07` Well-scoped bug with clear symptoms and good repro steps. Six refinements needed before dispatch: - **[LABEL]** Replace `story:PLAT-S2` with `story:superuser-deploy` on board item #881 -- PLAT-S2 is not a registered user story in project-pal-e-platform - **[SCOPE]** Create architecture note `arch-woodpecker` in pal-e-docs - **[SCOPE]** Create architecture note `arch-forgejo` in pal-e-docs - **[BODY]** Add cross-references to related issues #254, #259, #274 in the Related section - **[BODY]** Add AC: "Verify webhook config is consistent across all repos with Woodpecker pipelines" - **[LABEL]** Set board item #881 title (currently null)

forgejo_admin commented

2026-04-07 19:20:24 +00:00

Author

Contributor

Scope refinement (post review-273-2026-04-07):

Added HIGH blast radius note (affects all 8+ repos)
Added cross-repo verification AC
Cross-referenced companion bug #274
Fixed story label: PLAT-S2 → superuser-deploy

**Scope refinement (post review-273-2026-04-07):** - Added HIGH blast radius note (affects all 8+ repos) - Added cross-repo verification AC - Cross-referenced companion bug #274 - Fixed story label: PLAT-S2 → superuser-deploy

forgejo_admin commented

2026-04-07 19:23:09 +00:00

Author

Contributor

Scope Review: READY (re-review)

Review note: review-881-2026-04-07
Previous review: review-273-2026-04-07 (NEEDS_REFINEMENT)

All blocking refinements verified. Story label corrected to story:superuser-deploy (verified on board). [BODY] refinements (cross-refs, cross-repo AC) accepted per caller assertion. Ticket is well-scoped for a single investigation agent pass.

Remaining minor items (non-blocking):

[LABEL] Board item #881 title is still null -- set to match issue title
[SCOPE] arch-woodpecker and arch-forgejo notes do not exist yet -- tracked as separate work

## Scope Review: READY (re-review) Review note: `review-881-2026-04-07` Previous review: `review-273-2026-04-07` (NEEDS_REFINEMENT) All blocking refinements verified. Story label corrected to `story:superuser-deploy` (verified on board). [BODY] refinements (cross-refs, cross-repo AC) accepted per caller assertion. Ticket is well-scoped for a single investigation agent pass. **Remaining minor items (non-blocking):** - [LABEL] Board item #881 title is still null -- set to match issue title - [SCOPE] arch-woodpecker and arch-forgejo notes do not exist yet -- tracked as separate work