[BUG] verifyKeycloakJwt rejects valid tokens — Keycloak default access token aud=account, not westside-admin #26

Closed
opened 2026-05-03 21:23:28 +00:00 by forgejo_admin · 1 comment

Type

Bug

Lineage

Discovered during the same end-to-end SSO validation that surfaced #24 (cookie-too-big). After PR #25 fixed the cookie size (3151 B vs 4096 B cap, verified live), the SSO round-trip STILL produces an infinite redirect loop. Root cause is independent of #24 and unmasked only after #24 was fixed: the per-request JWT verification fails because the access token's aud claim is account (Keycloak default), not westside-admin.

Repo

forgejo_admin/westside-admin

What Broke

Same surface symptom as #24 (login → callback succeeds → next request to / redirects to /auth/login → infinite loop), but a different root cause that was masked by the cookie-size bug. After the cookie-size fix:

  1. Cookie IS set (3151 B, under the 4096 cap — verified via curl + Set-Cookie inspection).
  2. Cookie IS sent on the next request (verified — cookie jar carries it).
  3. Hook reads + decrypts the cookie successfully.
  4. verifyKeycloakJwt(access_token) throws JWTClaimValidationFailed because aud === 'account' but expectedAudience() === 'westside-admin'.
  5. Hook's catch {} block treats verify failure as anonymous → 302 to /auth/login.

Decoded the live access token claims (Lucas's session, post-#25 deploy at SHA 785a1ca3):

iss: https://keycloak.tail5b443a.ts.net/realms/westside-basketball   ✓ matches expectedIssuer
aud: account                                                          ✗ does NOT match expected 'westside-admin'
azp: westside-admin                                                   (authorized party — correct, but not what jose checks)
sub: 19bfe0df-7fbc-463c-97df-59d77901421e
typ: Bearer
realm_access.roles: [..., admin, ...]                                ✓ admin role present (but never gets checked, hook bails on aud)

Why this is a Keycloak default, not a misconfiguration

Per Keycloak documentation and OIDC convention: the access token's aud claim is populated from resource servers / audience mappers, not the requesting client. By default a client has zero protocol mappers and the token's aud becomes whatever scope-defined audiences exist (typically account from the standard account client scope). The requesting client's identity goes in the azp (authorized party) claim instead.

This is a well-known Keycloak gotcha. There are two industry-standard fixes:

A) Configure an Audience protocol mapper on the Keycloak westside-admin client that adds westside-admin to aud. Keeps the static-analysis-friendly audience: expectedAudience() invariant in jwtVerify. Requires a Keycloak admin-console click (no code change). Should be added to sop-keycloak-client-creation step 5 or 6 if we go this route.

B) Update verifyKeycloakJwt to accept azp === clientId in addition to (or instead of) aud === clientId. OIDC Core §2 defines azp as exactly "the party to which the ID Token was issued"; using it for client identity is RFC-compliant. Requires a code change in src/lib/server/keycloak.ts. No SOP change needed (works for any future Keycloak client without per-client mapper config).

Proposed Fix: B (code change)

Soften verifyKeycloakJwt to also accept azp matching the expected client_id. Concretely:

// In keycloak.ts, around verifyKeycloakJwt:
const result = await jwtVerify(token, jwks, {
  issuer: expectedIssuer()
  // remove audience: expectedAudience() — manual check below instead
});
const expectedClient = requireEnv('KEYCLOAK_CLIENT_ID');
const aud = result.payload.aud;
const audienceOk =
  (typeof aud === 'string' && aud === expectedClient) ||
  (Array.isArray(aud) && aud.includes(expectedClient));
const azpOk = result.payload.azp === expectedClient;
if (!audienceOk && !azpOk) {
  throw new JwtVerificationError(`claim aud/azp invalid (aud=${aud}, azp=${result.payload.azp})`);
}

Rationale for accepting both:

  • Accepts the Keycloak default (azp carries client_id, aud does not).
  • Stays correct if a future operator adds an audience mapper (then aud includes the client_id).
  • OIDC-spec-conformant for both regimes.

The unit test for verifyKeycloakJwt rejecting wrong-audience tokens (per keycloak.test.ts from PR #18) needs to be updated — its current test asserts rejection on aud mismatch alone, which would now incorrectly pass if azp matches. New cases:

  • aud=account, azp=westside-admin → ACCEPT (this is the Keycloak default, the bug we're fixing).
  • aud=other, azp=other → REJECT (truly wrong client).
  • aud=westside-admin, azp=other → ACCEPT (audience mapper configured, also valid).
  • aud=other, azp=westside-admin → ACCEPT (Keycloak default, also valid).
  • aud=account, azp=other → REJECT (no match).

Repro Steps

  1. Log in as a westside-basketball realm user with the admin role at https://westside-admin.tail5b443a.ts.net (post-PR-#25 image).
  2. Observe redirect loop ending in ERR_TOO_MANY_REDIRECTS (same surface symptom as #24).
  3. Decrypt the westside_admin_session cookie via COOKIE_SIGNING_KEY (kubectl -n westside-admin get secret westside-admin-secrets -o jsonpath='{.data.COOKIE_SIGNING_KEY}' | base64 -d), decode the embedded access_token's middle segment as JWT claims, observe aud = "account" and azp = "westside-admin".
  4. Cross-reference the hook's verifyKeycloakJwt call (line ~283 in keycloak.ts at SHA 785a1ca3): the audience: expectedAudience() arg to jwtVerify causes JWTClaimValidationFailed because "account" !== "westside-admin".

Expected Behavior

  • Admin user logs in once → cookie set → next request to / returns 200 with the admin app rendered, event.locals.user.name === 'draneylucas@gmail.com', event.locals.user.roles includes 'admin'.
  • Non-admin user logs in once → cookie set → next request renders the (unauthorized) 403 page with sign-out button.

Environment

  • Cluster: westside-admin namespace
  • Image SHA: 785a1ca311c1c964e26319c94373688f9ed1dfb1 (PR #25 merge — has the cookie-size fix; reproduces this aud bug consistently because the cookie now actually reaches the hook).
  • Keycloak westside-admin client UUID: c5749fa6-4d1e-4b07-bdc0-e371bf65e1e5
  • Keycloak client default scopes: [web-origins, acr, profile, roles, basic, email] — none of which include an audience mapper for westside-admin.

Acceptance Criteria

  • After deploying the fix, the SSO round-trip completes: GET / (no cookie) → 302 chain → eventual GET / returns 200 with admin app rendered.
  • Live curl flow from the issue body's "Repro Steps" no longer loops; / returns 200 after the callback's Set-Cookie.
  • Browser (Chromium via Playwright) accepts the cookie and renders the admin app at / with event.locals.user.name === 'draneylucas@gmail.com'.
  • Existing keycloak.test.ts JWT-verify cases still pass (no regression on truly-wrong-audience rejection).
  • New unit cases added per Proposed Fix → ACCEPT/REJECT matrix above; npm test exits 0.
  • No tokens / cookie ciphertext logged (grep verified).
  • PR body includes funnel-auth review per feedback_funnel_requires_auth explicitly addressing why accepting azp is OIDC-spec-conformant and not a security regression.
  • project-westside-admin
  • forgejo_admin/westside-admin#14 — original keycloak.ts (PR #18, the lib whose verification is too strict)
  • forgejo_admin/westside-admin#15 — hook (PR #20, downstream consumer of the verify failure)
  • forgejo_admin/westside-admin#22 — open follow-up touching the same lib (refresh_exp); coordinate via PR description, no semantic conflict
  • forgejo_admin/westside-admin#24 — sibling bug in the same validation pass (cookie too big — fixed in PR #25)
  • feedback_validate_before_done — drove this discovery
  • feedback_funnel_requires_auth — without this fix, the funnel-auth gate is non-functional even though every individual component is "approved"
  • sop-keycloak-client-creation — should consider documenting either the audience-mapper option (path A) or the azp-acceptance pattern (path B) so future clients don't repeat this
### Type Bug ### Lineage Discovered during the same end-to-end SSO validation that surfaced #24 (cookie-too-big). After PR #25 fixed the cookie size (3151 B vs 4096 B cap, verified live), the SSO round-trip STILL produces an infinite redirect loop. Root cause is independent of #24 and unmasked only after #24 was fixed: the per-request JWT verification fails because the access token's `aud` claim is `account` (Keycloak default), not `westside-admin`. ### Repo `forgejo_admin/westside-admin` ### What Broke Same surface symptom as #24 (login → callback succeeds → next request to `/` redirects to `/auth/login` → infinite loop), but a different root cause that was masked by the cookie-size bug. After the cookie-size fix: 1. Cookie IS set (3151 B, under the 4096 cap — verified via curl + Set-Cookie inspection). 2. Cookie IS sent on the next request (verified — cookie jar carries it). 3. Hook reads + decrypts the cookie successfully. 4. **`verifyKeycloakJwt(access_token)` throws `JWTClaimValidationFailed` because `aud === 'account'` but `expectedAudience() === 'westside-admin'`.** 5. Hook's `catch {}` block treats verify failure as anonymous → 302 to `/auth/login`. Decoded the live access token claims (Lucas's session, post-#25 deploy at SHA `785a1ca3`): ``` iss: https://keycloak.tail5b443a.ts.net/realms/westside-basketball ✓ matches expectedIssuer aud: account ✗ does NOT match expected 'westside-admin' azp: westside-admin (authorized party — correct, but not what jose checks) sub: 19bfe0df-7fbc-463c-97df-59d77901421e typ: Bearer realm_access.roles: [..., admin, ...] ✓ admin role present (but never gets checked, hook bails on aud) ``` ### Why this is a Keycloak default, not a misconfiguration Per Keycloak documentation and OIDC convention: the access token's `aud` claim is populated from **resource servers / audience mappers**, not the requesting client. By default a client has zero protocol mappers and the token's `aud` becomes whatever scope-defined audiences exist (typically `account` from the standard `account` client scope). The requesting client's identity goes in the `azp` (authorized party) claim instead. This is a well-known Keycloak gotcha. There are two industry-standard fixes: A) **Configure an Audience protocol mapper on the Keycloak `westside-admin` client** that adds `westside-admin` to `aud`. Keeps the static-analysis-friendly `audience: expectedAudience()` invariant in `jwtVerify`. **Requires a Keycloak admin-console click** (no code change). Should be added to `sop-keycloak-client-creation` step 5 or 6 if we go this route. B) **Update `verifyKeycloakJwt` to accept `azp === clientId`** in addition to (or instead of) `aud === clientId`. OIDC Core §2 defines `azp` as exactly "the party to which the ID Token was issued"; using it for client identity is RFC-compliant. **Requires a code change** in `src/lib/server/keycloak.ts`. No SOP change needed (works for any future Keycloak client without per-client mapper config). ### Proposed Fix: B (code change) Soften `verifyKeycloakJwt` to also accept `azp` matching the expected client_id. Concretely: ```ts // In keycloak.ts, around verifyKeycloakJwt: const result = await jwtVerify(token, jwks, { issuer: expectedIssuer() // remove audience: expectedAudience() — manual check below instead }); const expectedClient = requireEnv('KEYCLOAK_CLIENT_ID'); const aud = result.payload.aud; const audienceOk = (typeof aud === 'string' && aud === expectedClient) || (Array.isArray(aud) && aud.includes(expectedClient)); const azpOk = result.payload.azp === expectedClient; if (!audienceOk && !azpOk) { throw new JwtVerificationError(`claim aud/azp invalid (aud=${aud}, azp=${result.payload.azp})`); } ``` Rationale for accepting both: - Accepts the Keycloak default (`azp` carries client_id, `aud` does not). - Stays correct if a future operator adds an audience mapper (then `aud` includes the client_id). - OIDC-spec-conformant for both regimes. The unit test for `verifyKeycloakJwt` rejecting wrong-audience tokens (per `keycloak.test.ts` from PR #18) needs to be updated — its current test asserts rejection on `aud` mismatch alone, which would now incorrectly pass if `azp` matches. New cases: - `aud=account, azp=westside-admin` → ACCEPT (this is the Keycloak default, the bug we're fixing). - `aud=other, azp=other` → REJECT (truly wrong client). - `aud=westside-admin, azp=other` → ACCEPT (audience mapper configured, also valid). - `aud=other, azp=westside-admin` → ACCEPT (Keycloak default, also valid). - `aud=account, azp=other` → REJECT (no match). ### Repro Steps 1. Log in as a westside-basketball realm user with the `admin` role at `https://westside-admin.tail5b443a.ts.net` (post-PR-#25 image). 2. Observe redirect loop ending in `ERR_TOO_MANY_REDIRECTS` (same surface symptom as #24). 3. Decrypt the `westside_admin_session` cookie via `COOKIE_SIGNING_KEY` (`kubectl -n westside-admin get secret westside-admin-secrets -o jsonpath='{.data.COOKIE_SIGNING_KEY}' | base64 -d`), decode the embedded `access_token`'s middle segment as JWT claims, observe `aud = "account"` and `azp = "westside-admin"`. 4. Cross-reference the hook's `verifyKeycloakJwt` call (line ~283 in `keycloak.ts` at SHA `785a1ca3`): the `audience: expectedAudience()` arg to `jwtVerify` causes `JWTClaimValidationFailed` because `"account" !== "westside-admin"`. ### Expected Behavior - Admin user logs in once → cookie set → next request to `/` returns 200 with the admin app rendered, `event.locals.user.name === 'draneylucas@gmail.com'`, `event.locals.user.roles` includes `'admin'`. - Non-admin user logs in once → cookie set → next request renders the `(unauthorized)` 403 page with sign-out button. ### Environment - Cluster: `westside-admin` namespace - Image SHA: `785a1ca311c1c964e26319c94373688f9ed1dfb1` (PR #25 merge — has the cookie-size fix; reproduces this aud bug consistently because the cookie now actually reaches the hook). - Keycloak `westside-admin` client UUID: `c5749fa6-4d1e-4b07-bdc0-e371bf65e1e5` - Keycloak client default scopes: `[web-origins, acr, profile, roles, basic, email]` — none of which include an audience mapper for `westside-admin`. ### Acceptance Criteria - [ ] After deploying the fix, the SSO round-trip completes: GET `/` (no cookie) → 302 chain → eventual GET `/` returns 200 with admin app rendered. - [ ] Live curl flow from the issue body's "Repro Steps" no longer loops; `/` returns 200 after the callback's Set-Cookie. - [ ] Browser (Chromium via Playwright) accepts the cookie and renders the admin app at `/` with `event.locals.user.name === 'draneylucas@gmail.com'`. - [ ] Existing `keycloak.test.ts` JWT-verify cases still pass (no regression on truly-wrong-audience rejection). - [ ] New unit cases added per Proposed Fix → ACCEPT/REJECT matrix above; `npm test` exits 0. - [ ] No tokens / cookie ciphertext logged (grep verified). - [ ] PR body includes funnel-auth review per `feedback_funnel_requires_auth` explicitly addressing why accepting `azp` is OIDC-spec-conformant and not a security regression. ### Related - `project-westside-admin` - `forgejo_admin/westside-admin#14` — original keycloak.ts (PR #18, the lib whose verification is too strict) - `forgejo_admin/westside-admin#15` — hook (PR #20, downstream consumer of the verify failure) - `forgejo_admin/westside-admin#22` — open follow-up touching the same lib (refresh_exp); coordinate via PR description, no semantic conflict - `forgejo_admin/westside-admin#24` — sibling bug in the same validation pass (cookie too big — fixed in PR #25) - `feedback_validate_before_done` — drove this discovery - `feedback_funnel_requires_auth` — without this fix, the funnel-auth gate is non-functional even though every individual component is "approved" - `sop-keycloak-client-creation` — should consider documenting either the audience-mapper option (path A) or the azp-acceptance pattern (path B) so future clients don't repeat this
Author
Owner

Scope Review: APPROVED

Review note: review-1141-2026-05-03

Bug template fully populated. Live JWT claims + OIDC-convention rationale + accept/reject test matrix all in body. File targets verified at deployed SHA 785a1ca3: src/lib/server/keycloak.ts (verifyKeycloakJwt at line 283, offending audience check at line 293) and src/lib/server/keycloak.test.ts (existing aud-rejection test at line 212). Story label admin-row-crud verified on project page. arch:westside-admin backed by 3 project-specific arch notes per established convention. arch:keycloak gap waived per 10+ prior precedents. No sibling fanout (only westside-admin uses this jwtVerify pattern). 5-minute rule passes. No decomposition needed.

Two non-blocking [BODY] refinement notes for the dev:

  • signJwt test helper at line 74 doesn't currently support an azp parameter — dev will need to extend JwtOpts or use raw SignJWT for the new matrix cases.
  • Body's proposed snippet uses requireEnv('KEYCLOAK_CLIENT_ID') directly; existing expectedAudience() helper at line 126 does the same thing (cosmetic preference).

Two [SCOPE] follow-ups (NOT blocking on #26):

  • arch-keycloak note still missing in pal-e-docs (recurring platform gap; separate doc ticket).
  • sop-keycloak-client-creation should document either audience-mapper (path A) or azp-acceptance (path B) so future clients don't repeat this gotcha.

Ticket is scope-ready for todo. Dev agent can ship as-written.

## Scope Review: APPROVED Review note: `review-1141-2026-05-03` Bug template fully populated. Live JWT claims + OIDC-convention rationale + accept/reject test matrix all in body. File targets verified at deployed SHA `785a1ca3`: `src/lib/server/keycloak.ts` (verifyKeycloakJwt at line 283, offending audience check at line 293) and `src/lib/server/keycloak.test.ts` (existing aud-rejection test at line 212). Story label `admin-row-crud` verified on project page. `arch:westside-admin` backed by 3 project-specific arch notes per established convention. `arch:keycloak` gap waived per 10+ prior precedents. No sibling fanout (only westside-admin uses this jwtVerify pattern). 5-minute rule passes. No decomposition needed. Two non-blocking `[BODY]` refinement notes for the dev: - `signJwt` test helper at line 74 doesn't currently support an `azp` parameter — dev will need to extend `JwtOpts` or use raw `SignJWT` for the new matrix cases. - Body's proposed snippet uses `requireEnv('KEYCLOAK_CLIENT_ID')` directly; existing `expectedAudience()` helper at line 126 does the same thing (cosmetic preference). Two `[SCOPE]` follow-ups (NOT blocking on #26): - `arch-keycloak` note still missing in pal-e-docs (recurring platform gap; separate doc ticket). - `sop-keycloak-client-creation` should document either audience-mapper (path A) or azp-acceptance (path B) so future clients don't repeat this gotcha. Ticket is scope-ready for `todo`. Dev agent can ship as-written.
Sign in to join this conversation.
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
forgejo_admin/westside-admin#26
No description provided.