Bug: Gmail OAuth token refresh crashes on read-only k8s secret mount #136

New issue

Closed

opened 2026-03-21 20:38:35 +00:00 by forgejo_admin · 0 comments

forgejo_admin commented

2026-03-21 20:38:35 +00:00

Owner

Type

Bug

Lineage

plan-wkq → Phase 11 (Girls Tryout — March 24)
Discovered testing password reset flow (basketball-api #132)

Repo

forgejo_admin/basketball-api

What Broke

All Gmail OAuth email sending crashes after the first access token expires (~60 min). The gmail_sdk refreshes the token via the refresh token, then tries to write the new access token back to /secrets/google-oauth/gmail-westsidebasketball.json. This path is a k8s secret volume mounted read-only.

OSError: [Errno 30] Read-only file system: '/secrets/google-oauth/gmail-westsidebasketball.json'

This blocks: password reset emails, registration confirmations, roster exports, tryout announcements — ALL email.

Repro Steps

Wait for the Gmail access token to expire (~60 min after pod start)
Trigger any email-sending endpoint (e.g. POST /api/password-reset/request)
gmail_sdk refreshes access token successfully
gmail_sdk tries to save refreshed token to disk
OSError: Read-only file system → 500 Internal Server Error

Expected Behavior

Email sends successfully. Token refresh is transparent to the application.

Environment

Cluster/namespace: basketball-api
Pod: basketball-api deployment
Volume: gmail-oauth secret mounted at /secrets/google-oauth (readOnly: true)
SDK: gmail_sdk (gmail_sdk.auth._save_token)

Acceptance Criteria

Gmail emails send successfully even after access token expires
Token refresh + save works without crashing
Pod restarts don't break email (refresh token is preserved)
Password reset flow works end-to-end

Fix

EmptyDir + init container (standard k8s pattern):

Add init container that copies secret data to a writable emptyDir volume
App container mounts the writable emptyDir instead of the read-only secret
SDK writes refreshed tokens to the writable volume
On pod restart, init container re-copies from the immutable secret

File Targets

k8s/deployment.yaml — add init container + emptyDir volume

basketball-api #132 — password reset flow (blocked by this)
feedback_gmail_oauth_not_smtp.md — all email = Gmail OAuth
gmail_sdk auth.py:151 — the _save_token call that crashes

### Type Bug ### Lineage `plan-wkq` → Phase 11 (Girls Tryout — March 24) Discovered testing password reset flow (basketball-api #132) ### Repo `forgejo_admin/basketball-api` ### What Broke All Gmail OAuth email sending crashes after the first access token expires (~60 min). The gmail_sdk refreshes the token via the refresh token, then tries to write the new access token back to `/secrets/google-oauth/gmail-westsidebasketball.json`. This path is a k8s secret volume mounted read-only. ``` OSError: [Errno 30] Read-only file system: '/secrets/google-oauth/gmail-westsidebasketball.json' ``` This blocks: password reset emails, registration confirmations, roster exports, tryout announcements — ALL email. ### Repro Steps 1. Wait for the Gmail access token to expire (~60 min after pod start) 2. Trigger any email-sending endpoint (e.g. `POST /api/password-reset/request`) 3. gmail_sdk refreshes access token successfully 4. gmail_sdk tries to save refreshed token to disk 5. OSError: Read-only file system → 500 Internal Server Error ### Expected Behavior Email sends successfully. Token refresh is transparent to the application. ### Environment - Cluster/namespace: basketball-api - Pod: basketball-api deployment - Volume: `gmail-oauth` secret mounted at `/secrets/google-oauth` (readOnly: true) - SDK: gmail_sdk (`gmail_sdk.auth._save_token`) ### Acceptance Criteria - [ ] Gmail emails send successfully even after access token expires - [ ] Token refresh + save works without crashing - [ ] Pod restarts don't break email (refresh token is preserved) - [ ] Password reset flow works end-to-end ### Fix **EmptyDir + init container** (standard k8s pattern): - Add init container that copies secret data to a writable emptyDir volume - App container mounts the writable emptyDir instead of the read-only secret - SDK writes refreshed tokens to the writable volume - On pod restart, init container re-copies from the immutable secret ### File Targets - `k8s/deployment.yaml` — add init container + emptyDir volume ### Related - basketball-api #132 — password reset flow (blocked by this) - `feedback_gmail_oauth_not_smtp.md` — all email = Gmail OAuth - gmail_sdk `auth.py:151` — the `_save_token` call that crashes