Add DORA metrics exporter service

forgejo_admin commented

2026-03-02 05:24:18 +00:00

Contributor

Summary

Initial scaffold of the DORA metrics exporter -- a FastAPI service that polls Woodpecker CI and Forgejo APIs on a configurable interval and exposes DORA-aligned Prometheus metrics at /metrics. This is Phase 1 of the plan-2026-03-01-dora-metrics-dashboard.

Woodpecker collector: deployment frequency, pipeline duration, last success timestamp
Forgejo collector: PR merge count, PR lead time, review count per PR
Dockerfile with Forgejo PyPI --extra-index-url for SDK dependencies
Woodpecker CI pipeline to build and push image to Harbor

Changes

src/main.py -- FastAPI app with /health and /metrics endpoints, lifespan-managed background collector tasks
src/config.py -- Environment variable configuration (WOODPECKER_URL, WOODPECKER_TOKEN, FORGEJO_URL, FORGEJO_USER, FORGEJO_PASSWORD, POLL_INTERVAL)
src/collectors/woodpecker.py -- Polls Woodpecker list_repos + list_pipelines, updates dora_deployments_total (Counter), dora_deployment_duration_seconds (Histogram), dora_deployment_last_success_timestamp (Gauge)
src/collectors/forgejo.py -- Polls Forgejo repo_search + repo_list_pull_requests + repo_list_pull_reviews, updates dora_pr_merges_total (Counter), dora_pr_lead_time_seconds (Histogram), dora_pr_review_count (Histogram)
Dockerfile -- Python 3.12-slim, installs from Forgejo PyPI, runs uvicorn on port 8000
.woodpecker.yaml -- CI pipeline builds Docker image and pushes to Harbor on push to main
requirements.txt -- fastapi, uvicorn, prometheus-client, ldraney-woodpecker-sdk, ldraney-forgejo-sdk
.gitignore -- Python standard ignores
README.md -- Metrics reference, configuration table, run instructions

Test Plan

Python syntax check passes on all source files (python -m py_compile)
Container builds successfully: docker build -t dora-exporter .
Health endpoint returns 200: curl localhost:8000/health
Metrics endpoint returns Prometheus format: curl localhost:8000/metrics
Woodpecker pipeline builds and pushes to Harbor after merge

Review Checklist

SDK import names match published packages (woodpecker_sdk, forgejo_sdk)
Synchronous SDK calls run in executor (not blocking event loop)
Counters use seen-ID tracking to avoid double-counting
Histograms observe each item once via seen-ID sets
Gauge tracks max timestamp per repo in local dict (no prometheus_client internals)
Forgejo collector handles both list and dict returns from repo_search
ISO-8601 timestamp parsing handles both "Z" and "+00:00" suffixes
Graceful shutdown cancels tasks and closes SDK clients

plan-2026-03-01-dora-metrics-dashboard -- parent plan
issue-pal-e-dora-exporter-service -- tracking issue
dora-framework -- the DORA axiom this implements

## Summary Initial scaffold of the DORA metrics exporter -- a FastAPI service that polls Woodpecker CI and Forgejo APIs on a configurable interval and exposes DORA-aligned Prometheus metrics at `/metrics`. This is Phase 1 of the `plan-2026-03-01-dora-metrics-dashboard`. - Woodpecker collector: deployment frequency, pipeline duration, last success timestamp - Forgejo collector: PR merge count, PR lead time, review count per PR - Dockerfile with Forgejo PyPI `--extra-index-url` for SDK dependencies - Woodpecker CI pipeline to build and push image to Harbor ## Changes - `src/main.py` -- FastAPI app with `/health` and `/metrics` endpoints, lifespan-managed background collector tasks - `src/config.py` -- Environment variable configuration (WOODPECKER_URL, WOODPECKER_TOKEN, FORGEJO_URL, FORGEJO_USER, FORGEJO_PASSWORD, POLL_INTERVAL) - `src/collectors/woodpecker.py` -- Polls Woodpecker `list_repos` + `list_pipelines`, updates `dora_deployments_total` (Counter), `dora_deployment_duration_seconds` (Histogram), `dora_deployment_last_success_timestamp` (Gauge) - `src/collectors/forgejo.py` -- Polls Forgejo `repo_search` + `repo_list_pull_requests` + `repo_list_pull_reviews`, updates `dora_pr_merges_total` (Counter), `dora_pr_lead_time_seconds` (Histogram), `dora_pr_review_count` (Histogram) - `Dockerfile` -- Python 3.12-slim, installs from Forgejo PyPI, runs uvicorn on port 8000 - `.woodpecker.yaml` -- CI pipeline builds Docker image and pushes to Harbor on push to main - `requirements.txt` -- fastapi, uvicorn, prometheus-client, ldraney-woodpecker-sdk, ldraney-forgejo-sdk - `.gitignore` -- Python standard ignores - `README.md` -- Metrics reference, configuration table, run instructions ## Test Plan - [ ] Python syntax check passes on all source files (`python -m py_compile`) - [ ] Container builds successfully: `docker build -t dora-exporter .` - [ ] Health endpoint returns 200: `curl localhost:8000/health` - [ ] Metrics endpoint returns Prometheus format: `curl localhost:8000/metrics` - [ ] Woodpecker pipeline builds and pushes to Harbor after merge ## Review Checklist - [x] SDK import names match published packages (`woodpecker_sdk`, `forgejo_sdk`) - [x] Synchronous SDK calls run in executor (not blocking event loop) - [x] Counters use seen-ID tracking to avoid double-counting - [x] Histograms observe each item once via seen-ID sets - [x] Gauge tracks max timestamp per repo in local dict (no prometheus_client internals) - [x] Forgejo collector handles both list and dict returns from `repo_search` - [x] ISO-8601 timestamp parsing handles both "Z" and "+00:00" suffixes - [x] Graceful shutdown cancels tasks and closes SDK clients ## Related Notes - `plan-2026-03-01-dora-metrics-dashboard` -- parent plan - `issue-pal-e-dora-exporter-service` -- tracking issue - `dora-framework` -- the DORA axiom this implements

forgejo_admin added 1 commit

2026-03-02 05:24:18 +00:00

Add DORA metrics exporter service b5f97b7489

FastAPI service that polls Woodpecker CI and Forgejo APIs, exposing
DORA metrics (deployment frequency, lead time, change failure rate
proxies) as Prometheus endpoints at /metrics.

Collectors:
- Woodpecker: pipeline counts, durations, last success timestamps
- Forgejo: PR merge counts, lead times, review counts

Includes Dockerfile with Forgejo PyPI index for SDK deps, and
Woodpecker CI pipeline for Harbor image builds.

Issue: issue-pal-e-dora-exporter-service
Plan: plan-2026-03-01-dora-metrics-dashboard (Phase 1)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

forgejo_admin added 1 commit

2026-03-02 05:24:58 +00:00

Fix: use asyncio.get_running_loop() instead of deprecated get_event_loop() 04d283ea22

get_event_loop() is deprecated in Python 3.10+ and emits a warning
when called from within a running async context. get_running_loop()
is the correct replacement for code that always runs inside an
async task.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

forgejo_admin commented

2026-03-02 05:25:07 +00:00

Author

Contributor

Review-fix round 1:

Fixed asyncio.get_event_loop() -> asyncio.get_running_loop() in both collectors. The former is deprecated in Python 3.10+ and emits a DeprecationWarning when called from within an already-running event loop. Since _run_sync() is always called from async task context, get_running_loop() is the correct API.

**Review-fix round 1:** Fixed `asyncio.get_event_loop()` -> `asyncio.get_running_loop()` in both collectors. The former is deprecated in Python 3.10+ and emits a DeprecationWarning when called from within an already-running event loop. Since `_run_sync()` is always called from async task context, `get_running_loop()` is the correct API.