feat: add pipeline failure rate tool for DORA CFR/MTTR #5
No reviewers
Labels
No labels
domain:backend
domain:devops
domain:frontend
status:approved
status:in-progress
status:needs-fix
status:qa
type:bug
type:devops
type:feature
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
ldraney/woodpecker-mcp!5
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "4-pipeline-failure-data"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Summary
Add
get_pipeline_failure_rateMCP tool that aggregates pipeline success/failure counts over a configurable time window, enabling CFR and MTTR calculation from the agent side.Changes
src/woodpecker_mcp/tools/dora.py-- new DORA tool module withget_pipeline_failure_ratetool and_fetch_all_pipelinespagination helpersrc/woodpecker_mcp/tools/__init__.py-- register the dora moduletests/test_dora.py-- 11 unit tests with mocked SDK + 2 integration testsTest Plan
Related
🤖 Generated with Claude Code
PR #5 Review
DOMAIN REVIEW
Tech stack: Python / FastMCP / Pydantic / Woodpecker SDK -- same stack as the existing tool modules.
Pattern consistency with existing tools: The new
dora.pymodule follows the established patterns frompipelines.pyprecisely:from ..server import _error_response, _ok, get_client, lookup_repo, mcp@mcp.tool()decorator +Annotated[..., Field(...)]parameter style_error_response(exc)error handlinglookup_repo()->get_client()flow__init__.py-- correctPagination helper (
_fetch_all_pipelines): Well-structured, clean exit conditions (empty batch or partial page). Matches the pagination approach described in the existing codebase.Client-side filtering (lines 82-85): The code fetches ALL pipelines in the time range and then filters by branch/event in Python, whereas the existing
list_pipelinestool passes these filters to the SDK directly. The inline comment explains this is intentional ("keep the pagination logic simple"). For the first iteration this is acceptable, but note that repos with heavy pipeline traffic would fetch significantly more data than necessary.CFR calculation: Correct. Excludes pending/running/blocked/declined from the denominator (reported as
other_count), which is the right approach -- in-progress pipelines are not yet pass/fail.Failure classification:
killedcounted as failure alongsidefailureanderror-- reasonable and well-tested.BLOCKERS
None. The code is correct, follows established patterns, has solid test coverage, and introduces no security risks.
NITS
Unused
beforeparameter in_fetch_all_pipelines(line 17): The function signature acceptsbeforebut the caller on line 74 never passes it. The function calculatesnowinternally but only passesafter_str. Either passbeforefrom the caller for completeness, or remove it from the helper's signature to avoid misleading readers.No input validation on
daysparameter: A caller passingdays=0getsafter=now(empty window).days=-1setsafterto the future. Neither is dangerous, but adays < 1guard with a clear error message would be more user-friendly. Alternatively, PydanticField(ge=1)would catch this at the schema level.Sort key fallback (line 130):
lambda f: f.get("created_at") or 0-- theor 0also triggers oncreated_at=0(Unix epoch). This is practically impossible for real pipeline data, butf.get("created_at", 0)would be more precise and idiomatic.Missing error-path test: No unit test for what happens when
lookup_reporaises an exception. The existingpipelines.pytools rely on the same try/except pattern so it is implicitly tested elsewhere, but atest_error_handlingcase in this module would make it self-contained.Missing edge-case test for
recent_failure_limit=0: The limit is applied viafailures[:0]which returns an empty list -- correct behavior, but worth an explicit test to document the intent.Performance at scale: The client-side filtering approach means a repo with 10,000 pipelines over 30 days downloads all of them even when filtering for a single branch. For a v1 this is fine, but consider passing
branch/eventto the SDK in a follow-up when pagination logic is revisited.SOP COMPLIANCE
PROCESS OBSERVATIONS
__init__.py.get_pipeline_failure_rateclearly explains the status categorization and DORA metric intent. Good for agent consumption.side_effectfor multi-page responses is particularly solid. Integration tests appropriately usepytest.skipfor API errors.VERDICT: APPROVED
Clean implementation that follows established patterns. Nits are all non-blocking improvements. Ship it.
PR #5 Review
DOMAIN REVIEW
Tech stack: Python / FastMCP (MCP server) / Pydantic / pytest
New
get_pipeline_failure_ratetool insrc/woodpecker_mcp/tools/dora.pywith supporting pagination helper and comprehensive test suite intests/test_dora.py.Pattern compliance -- PASS. The new tool follows every established codebase convention:
@mcp.tool()decorator with-> strreturn typeAnnotated[type, Field(description="...")]parameter annotationsfrom ..server import _error_response, _ok, get_client, lookup_repo, mcpimport linefrom __future__ import annotationsat module top_error_response(exc)on failure,_ok(data)on success__init__.pyin alphabetical order with# noqa: F401Response structure quality -- GOOD. The output is well-designed for agent consumption:
counts.total,counts.success,counts.failure,counts.other-- clean aggregateschange_failure_rate-- pre-calculated CFR (saves agent from doing arithmetic)recent_failureswithcreated_at,started_at,finished_attimestamps -- enables MTTR calculationtime_rangeandfiltersechoed back -- self-documenting responseStatus categorization -- CORRECT. success={success}, failure={failure, error, killed}, excluded={pending, running, blocked, declined}. The
killed-> failure mapping is the right call for DORA metrics.Pagination helper
_fetch_all_pipelines-- clean implementation, consistent with patterns elsewhere in the codebase.Client-side filtering note: Branch/event filters are applied after fetching all pipelines. The inline comment explains the rationale (keeps pagination simple). This is fine for typical DORA windows (30-90 days). For repos with thousands of daily pipelines this could become slow, but that is a future optimization concern, not a blocker.
BLOCKERS
None.
NITS
Module docstring word count: The docstring says "pipeline failure rate and recent failures for CFR/MTTR calculation" which accurately describes the single tool. If more DORA tools land in this module later (e.g., deployment frequency, lead time), the docstring will need updating. No action needed now.
Sorting by
created_atwith fallback to0: Linefailures.sort(key=lambda f: f.get("created_at") or 0, reverse=True)-- theor 0fallback means pipelines withcreated_at=Nonesort to the end. This is correct behavior but worth noting: if Woodpecker ever returnsNoneforcreated_at, those entries will cluster at the tail of the list rather than raising. Acceptable.SOP COMPLIANCE
PROCESS OBSERVATIONS
VERDICT: APPROVED
PR #5 Review
Post-merge review of
feat: add pipeline failure rate tool for DORA CFR/MTTR.DOMAIN REVIEW
Stack: Python / FastMCP / Woodpecker SDK / pytest
Pattern Conformance -- STRONG
The new
dora.pymodule follows the established codebase patterns precisely:from __future__ import annotationsimport..server:_error_response,_ok,get_client,lookup_repo,mcp@mcp.tool()decorator withAnnotated[type, Field(description="...")]parameter annotationsstr(JSON-formatted)try/except Exceptionwrapping with_error_response(exc)for error handlinglookup_repo()used correctly for repo resolution__init__.pyin alphabetical orderAPI Design -- SOLID
_fetch_all_pipelinespagination helper is clean: terminates on empty batch or partial pageother_countfor pending/running/blocked/declined is a smart design choice -- keeps CFR denominator cleankilledclassified as failure is the correct DORA interpretationchange_failure_ratewithround(cfr, 4)saves agents from doing mathrecent_failureswith timestamps (created_at,started_at,finished_at) enables MTTR calculation downstreamTest Coverage -- COMPREHENSIVE (11 unit + 2 integration)
Unit tests cover:
recent_failure_limitcapkilledstatus classificationIntegration tests:
pytest.skipon API errors (graceful degradation)BLOCKERS
None.
NITS
Sorting by integer timestamps:
failures.sort(key=lambda f: f.get("created_at") or 0, reverse=True)--created_atfrom the Woodpecker API is a unix timestamp (int). Theor 0fallback handles None, but pipelines withcreated_at = 0would sort unpredictably among nulls. This is unlikely in practice but a(f.get("created_at") or 0)could be(f.get("created_at") or -1)to push nulls to the end. Purely cosmetic.No
beforeparameter passed to API:_fetch_all_pipelines(repo_id, after=after_str)is called withoutbefore, so technically it fetches everything after the start date including pipelines created after thenowsnapshot. For a tool called once this is fine, but if the tool were called in a tight loop with overlapping windows, the upper bound is slightly fuzzy. Not a real issue for DORA use cases.Module docstring specificity: Existing modules enumerate tool count (e.g.,
"Pipeline tools -- trigger, status, logs, restart, cancel, list (6 tools)"). The new module says"DORA tools -- pipeline failure rate and recent failures for CFR/MTTR calculation."-- this is descriptive but doesn't follow the(N tools)suffix convention. Consider:"DORA tools -- pipeline failure rate (1 tool).".Unit tests use
@patchdecorator stacking: The unit tests correctly mock bothget_clientandlookup_repo, but every test method repeats the same two@patchdecorators. A class-level@patchor a pytest fixture could reduce duplication. Not a blocker -- the current approach is clear and explicit.SOP COMPLIANCE
Closes #4not present in PR body (PR saysForgejo issue: #4but does not use theCloseskeyword for auto-close)refs/pull/5/head)PROCESS OBSERVATIONS
__init__.py.VERDICT: APPROVED