ldraney/pal-e-dora-exporter
1
0
Fork 0
Code Issues 1 Pull requests Projects Releases Packages Wiki Activity Actions

Close CFR gap: ArgoCD event integration for pal-e-deployments #6

New issue
Closed
opened 2026-06-13 18:34:08 +00:00 by ldraney · 6 comments
ldraney commented 2026-06-13 18:34:08 +00:00
Owner
Copy link

Type

Spike

Lineage

Migrated from ldraney/DORA#6.
Standalone — identified as measurement gap in DORA framework.

Repo

ldraney/pal-e-dora-exporter

User Story

As a platform operator
I want Tier 1 DORA composite in Grafana
So that I know if production pipelines are healthy without digging

Context

pal-e-deployments deploys via ArgoCD auto-sync, bypassing Woodpecker entirely. The exporter has no visibility into sync success/failure. Options: ArgoCD notifications webhook to exporter, or leverage ArgoCD's own Prometheus metrics (argocd_app_sync_total) if already exported.

No ArgoCD configuration was found in pal-e-platform, so it is currently unknown whether ArgoCD exports Prometheus metrics on this cluster.

Question

  1. Does ArgoCD already export argocd_app_sync_total to Prometheus on this cluster?
  2. If yes, can we query it directly in Grafana without building a new collector in pal-e-dora-exporter?
  3. If no, what is the minimal ArgoCD webhook/notification setup required to feed sync events to the exporter?

Deliverables

  • docs/argocd-cfr-decision.md documenting the chosen approach with rationale, tradeoffs evaluated, and any ArgoCD configuration changes required
  • Follow-up feature ticket(s) created on the appropriate repo based on the decision, with refined scope from spike findings. If no action needed, the docs file explains why.

Time-box

2 hours max. If ArgoCD metrics discovery takes longer, escalate to Lucas for a decision on whether to pursue webhook approach without full investigation.

Constraints

  • Evaluate whether ArgoCD already exports Prometheus metrics before building a new collector
  • Minimal changes to ArgoCD configuration preferred

Related

  • #7 — MTTR calculation depends on this spike's outcome (deployment event source determines MTTR measurement approach)
  • dora-metrics — project this affects
### Type Spike ### Lineage Migrated from ldraney/DORA#6. Standalone — identified as measurement gap in DORA framework. ### Repo `ldraney/pal-e-dora-exporter` ### User Story As a platform operator I want Tier 1 DORA composite in Grafana So that I know if production pipelines are healthy without digging ### Context pal-e-deployments deploys via ArgoCD auto-sync, bypassing Woodpecker entirely. The exporter has no visibility into sync success/failure. Options: ArgoCD notifications webhook to exporter, or leverage ArgoCD's own Prometheus metrics (`argocd_app_sync_total`) if already exported. No ArgoCD configuration was found in `pal-e-platform`, so it is currently unknown whether ArgoCD exports Prometheus metrics on this cluster. ### Question 1. Does ArgoCD already export `argocd_app_sync_total` to Prometheus on this cluster? 2. If yes, can we query it directly in Grafana without building a new collector in pal-e-dora-exporter? 3. If no, what is the minimal ArgoCD webhook/notification setup required to feed sync events to the exporter? ### Deliverables - [ ] `docs/argocd-cfr-decision.md` documenting the chosen approach with rationale, tradeoffs evaluated, and any ArgoCD configuration changes required - [ ] Follow-up feature ticket(s) created on the appropriate repo based on the decision, with refined scope from spike findings. If no action needed, the docs file explains why. ### Time-box 2 hours max. If ArgoCD metrics discovery takes longer, escalate to Lucas for a decision on whether to pursue webhook approach without full investigation. ### Constraints - Evaluate whether ArgoCD already exports Prometheus metrics before building a new collector - Minimal changes to ArgoCD configuration preferred ### Related - #7 — MTTR calculation depends on this spike's outcome (deployment event source determines MTTR measurement approach) - `dora-metrics` — project this affects
ldraney commented 2026-06-13 19:16:23 +00:00
Author
Owner
Copy link

Issue #6 Template Review

TEMPLATE CONFORMANCE

  • ### Type -- present, value "Feature" matches type:feature label
  • ### Lineage -- present, traces to ldraney/DORA#6
  • ### Repo -- present, correct repo
  • ### User Story -- present, follows As/I want/So that format
  • ### Context -- present, non-empty, explains the ArgoCD gap
  • ### File Targets -- present (see content quality notes below)
  • ### Feature Flag -- present, "None required" with justification
  • ### Acceptance Criteria -- present, uses - [ ] checkbox format
  • ### Test Expectations -- present (see content quality notes below)
  • ### Constraints -- present, non-empty
  • ### Checklist -- present, standard format
  • ### Related -- present

All 12 required sections are present and non-empty.

CONTENT QUALITY

File Targets -- ambiguous (BLOCKER)
The file targets present two alternative approaches ("if webhook approach" vs "if ArgoCD already exports to Prometheus") without committing to one. This is a feature ticket, not a spike. An agent picking this up will not know which path to implement. Either:

  1. Split this into a spike (issue to determine the approach) + a follow-up feature (with concrete file targets once approach is chosen), OR
  2. Commit to one approach and list specific files.

The path src/collectors/argocd.py is consistent with the existing repo structure (src/collectors/forgejo.py, src/collectors/woodpecker.py exist), so the webhook-approach target is plausible. But the Grafana dashboard alternative has no file path at all -- "Grafana dashboard queries" is not actionable for an agent.

File Targets -- missing "should NOT touch" list (nit)
Template calls for a "Files the agent should NOT touch" sub-section. Not present. Not a blocker but useful for scope guardrails.

Acceptance Criteria -- reasonable but couples to approach decision
The three criteria are testable and well-formed, but they assume the webhook/collector approach (they reference dora_deployments_total{repo="pal-e-deployments"}). If the alternative approach (direct ArgoCD Prometheus metrics in Grafana) is chosen, these criteria would not apply. This is downstream of the file targets ambiguity.

Test Expectations -- too thin
Only one integration test is listed. The template asks for both unit and integration tests. If a new argocd.py collector is built, it needs unit tests for the collector logic (mocked ArgoCD events) in addition to the integration test. Missing unit test expectations = the agent will likely skip them.

Context -- good
Clearly explains why pal-e-deployments is invisible to the exporter and names the two options. This is well-written.

User Story -- scope mismatch (nit)
The user story says "Tier 1 DORA composite in Grafana" but the acceptance criteria only address CFR for pal-e-deployments. The composite metric (combining DF, CFR, MTTR, CLT) is a larger scope than what this issue delivers. Consider narrowing the user story to match the actual deliverable: "I want CFR visibility for pal-e-deployments in Grafana."

BLOCKERS

  1. File Targets present two mutually exclusive approaches without choosing one. A feature ticket must have a concrete implementation path. Either resolve the approach question first (via a spike) or commit to one approach and update file targets accordingly.

NITS

  1. "Files the agent should NOT touch" sub-section missing from File Targets.
  2. User story scope ("Tier 1 DORA composite") is broader than what acceptance criteria actually deliver (CFR for pal-e-deployments).
  3. Test Expectations list only integration -- add unit test expectation for the new collector.

VERDICT: NEEDS_REWORK

The issue is well-structured and all template sections are present, but the core file targets section is ambiguous -- it describes two alternative implementation paths without committing to one. This makes it unsuitable for agent execution. Recommend either: (a) run a spike first to determine the approach, then update this ticket with concrete targets, or (b) pick the webhook collector approach, lock in file targets, and add unit test expectations.

## Issue #6 Template Review ### TEMPLATE CONFORMANCE - [x] `### Type` -- present, value "Feature" matches `type:feature` label - [x] `### Lineage` -- present, traces to ldraney/DORA#6 - [x] `### Repo` -- present, correct repo - [x] `### User Story` -- present, follows As/I want/So that format - [x] `### Context` -- present, non-empty, explains the ArgoCD gap - [x] `### File Targets` -- present (see content quality notes below) - [x] `### Feature Flag` -- present, "None required" with justification - [x] `### Acceptance Criteria` -- present, uses `- [ ]` checkbox format - [x] `### Test Expectations` -- present (see content quality notes below) - [x] `### Constraints` -- present, non-empty - [x] `### Checklist` -- present, standard format - [x] `### Related` -- present All 12 required sections are present and non-empty. ### CONTENT QUALITY **File Targets -- ambiguous (BLOCKER)** The file targets present two alternative approaches ("if webhook approach" vs "if ArgoCD already exports to Prometheus") without committing to one. This is a feature ticket, not a spike. An agent picking this up will not know which path to implement. Either: 1. Split this into a spike (issue to determine the approach) + a follow-up feature (with concrete file targets once approach is chosen), OR 2. Commit to one approach and list specific files. The path `src/collectors/argocd.py` is consistent with the existing repo structure (`src/collectors/forgejo.py`, `src/collectors/woodpecker.py` exist), so the webhook-approach target is plausible. But the Grafana dashboard alternative has no file path at all -- "Grafana dashboard queries" is not actionable for an agent. **File Targets -- missing "should NOT touch" list (nit)** Template calls for a "Files the agent should NOT touch" sub-section. Not present. Not a blocker but useful for scope guardrails. **Acceptance Criteria -- reasonable but couples to approach decision** The three criteria are testable and well-formed, but they assume the webhook/collector approach (they reference `dora_deployments_total{repo="pal-e-deployments"}`). If the alternative approach (direct ArgoCD Prometheus metrics in Grafana) is chosen, these criteria would not apply. This is downstream of the file targets ambiguity. **Test Expectations -- too thin** Only one integration test is listed. The template asks for both unit and integration tests. If a new `argocd.py` collector is built, it needs unit tests for the collector logic (mocked ArgoCD events) in addition to the integration test. Missing unit test expectations = the agent will likely skip them. **Context -- good** Clearly explains why pal-e-deployments is invisible to the exporter and names the two options. This is well-written. **User Story -- scope mismatch (nit)** The user story says "Tier 1 DORA composite in Grafana" but the acceptance criteria only address CFR for pal-e-deployments. The composite metric (combining DF, CFR, MTTR, CLT) is a larger scope than what this issue delivers. Consider narrowing the user story to match the actual deliverable: "I want CFR visibility for pal-e-deployments in Grafana." ### BLOCKERS 1. **File Targets present two mutually exclusive approaches without choosing one.** A feature ticket must have a concrete implementation path. Either resolve the approach question first (via a spike) or commit to one approach and update file targets accordingly. ### NITS 1. "Files the agent should NOT touch" sub-section missing from File Targets. 2. User story scope ("Tier 1 DORA composite") is broader than what acceptance criteria actually deliver (CFR for pal-e-deployments). 3. Test Expectations list only integration -- add unit test expectation for the new collector. ### VERDICT: NEEDS_REWORK The issue is well-structured and all template sections are present, but the core file targets section is ambiguous -- it describes two alternative implementation paths without committing to one. This makes it unsuitable for agent execution. Recommend either: (a) run a spike first to determine the approach, then update this ticket with concrete targets, or (b) pick the webhook collector approach, lock in file targets, and add unit test expectations.
ldraney commented 2026-06-13 19:20:33 +00:00
Author
Owner
Copy link

Converted to spike per QA review. The two implementation approaches (new webhook collector vs ArgoCD native Prometheus metrics) are mutually exclusive and need investigation before an agent can execute.

Additionally, no ArgoCD-related files were found in pal-e-platform, so we don't yet know whether ArgoCD metrics are even exposed on this cluster.

Spike deliverable: Decision comment on this issue recommending one approach with rationale.

Converted to spike per QA review. The two implementation approaches (new webhook collector vs ArgoCD native Prometheus metrics) are mutually exclusive and need investigation before an agent can execute. Additionally, no ArgoCD-related files were found in `pal-e-platform`, so we don't yet know whether ArgoCD metrics are even exposed on this cluster. **Spike deliverable:** Decision comment on this issue recommending one approach with rationale.
ldraney commented 2026-06-13 19:23:28 +00:00
Author
Owner
Copy link

Issue #6 Template Review (Re-review)

Previous review flagged ambiguous dual-approach scope. Issue was converted from Feature to Spike. Re-reviewing against template-issue-spike.

TEMPLATE CONFORMANCE

Required sections from template-issue-spike:

Required Section Status Notes
### Type PASS Correctly set to "Spike"
### Lineage PASS Migration context from ldraney/DORA#6 present
### Repo PASS ldraney/pal-e-dora-exporter
### Question FAIL Section is named ### Investigation Questions instead of ### Question. Content exists but does not follow template format -- should be a top-level framing question with bullet sub-questions, not a numbered list.
### Deliverables FAIL Section is named ### Spike Deliverables instead of ### Deliverables. Content does not match template requirements (see BLOCKERS below).
### Time-box FAIL Completely absent.
### Related PASS Cross-reference to #7 present with clear dependency rationale.

Extra sections not in spike template: ### User Story, ### Context, ### Constraints. These are not harmful but ### User Story is a Feature-template section -- spikes do not have user stories.

CONTENT QUALITY

What improved from the original Feature version:

  • The dual-approach ambiguity is resolved. The issue now correctly frames this as an investigation ("does ArgoCD already export metrics?") rather than prescribing an implementation.
  • The cross-reference to #7 (MTTR depends on deployment event source) is present and well-articulated.
  • The investigation questions are clear and logically ordered (check existing metrics first, then evaluate alternatives).

What still needs work:

  1. Question format: The template calls for a top-level framing question (yes/no or "which approach") with bullet sub-questions. Current format is a numbered list without a framing question. Should be restructured as:

    ### Question
Should pal-e-dora-exporter build a new ArgoCD collector, or can we
leverage existing ArgoCD Prometheus metrics for CFR measurement?
- Does ArgoCD already export `argocd_app_sync_total` on this cluster?
- If yes, can Grafana query it directly without a new collector?
- If no, what is the minimal webhook/notification setup required?

  2. Deliverables do not match template requirements. The template mandates two specific outputs:

    • A docs/{topic}.md file (the durable artifact, merged via docs-only PR)
    • Follow-up tickets created or updated with refined scope

    The current deliverables list a "decision document posted as a comment on this issue" -- this contradicts the template. Spike knowledge must live in a docs/ file, not an issue comment. Comments are ephemeral; docs files are durable.

  3. Time-box is missing entirely. Every spike must have a maximum time and an escalation clause ("if time-box expires without answer: close spike, document findings, escalate to Lucas for direction").

BLOCKERS

  1. Missing ### Time-box section -- Required by spike template. Without a time-box, spikes become open-ended investigations. Must include maximum time (e.g., "2 hours" or "1 session") and escalation clause.

  2. Deliverables do not include docs/ file -- Template requires docs/{topic}.md as a durable artifact. "Decision document posted as a comment" does not satisfy this. Change to: docs/argocd-cfr-integration.md (or similar).

  3. Deliverables do not include follow-up tickets -- Template requires "Follow-up tickets created or existing tickets updated with refined scope." This is the whole point of a spike: investigation produces scoped work items.

  4. Section headers do not match template -- ### Investigation Questions should be ### Question; ### Spike Deliverables should be ### Deliverables. Standardized headers enable tooling and agent parsing.

NITS

  • Remove ### User Story -- spikes do not have user stories per template guidance. The context section already covers the "why."
  • ### Context and ### Constraints are not in the template but contain useful information. Consider folding their content into the ### Question section as background context, or keep them as supplementary sections below the required ones.
  • The ### Related section uses dora-metrics as the project slug -- verify this matches the actual pal-e-docs project slug.

VERDICT: NOT APPROVED

The conversion from Feature to Spike was the right call -- the investigation framing is much better. However, three required spike template sections are missing or non-conformant: ### Time-box (absent), ### Deliverables (wrong outputs), and ### Question (wrong header and format). These are structural blockers per template conformance rules. Fix the section headers, add a time-box with escalation clause, and align deliverables to the two required spike outputs (docs/ file + follow-up tickets).

## Issue #6 Template Review (Re-review) Previous review flagged ambiguous dual-approach scope. Issue was converted from Feature to Spike. Re-reviewing against `template-issue-spike`. ### TEMPLATE CONFORMANCE Required sections from `template-issue-spike`: | Required Section | Status | Notes | |---|---|---| | `### Type` | PASS | Correctly set to "Spike" | | `### Lineage` | PASS | Migration context from `ldraney/DORA#6` present | | `### Repo` | PASS | `ldraney/pal-e-dora-exporter` | | `### Question` | FAIL | Section is named `### Investigation Questions` instead of `### Question`. Content exists but does not follow template format -- should be a top-level framing question with bullet sub-questions, not a numbered list. | | `### Deliverables` | FAIL | Section is named `### Spike Deliverables` instead of `### Deliverables`. Content does not match template requirements (see BLOCKERS below). | | `### Time-box` | FAIL | **Completely absent.** | | `### Related` | PASS | Cross-reference to #7 present with clear dependency rationale. | Extra sections not in spike template: `### User Story`, `### Context`, `### Constraints`. These are not harmful but `### User Story` is a Feature-template section -- spikes do not have user stories. ### CONTENT QUALITY **What improved from the original Feature version:** - The dual-approach ambiguity is resolved. The issue now correctly frames this as an investigation ("does ArgoCD already export metrics?") rather than prescribing an implementation. - The cross-reference to #7 (MTTR depends on deployment event source) is present and well-articulated. - The investigation questions are clear and logically ordered (check existing metrics first, then evaluate alternatives). **What still needs work:** 1. **Question format:** The template calls for a top-level framing question (yes/no or "which approach") with bullet sub-questions. Current format is a numbered list without a framing question. Should be restructured as: ``` ### Question Should pal-e-dora-exporter build a new ArgoCD collector, or can we leverage existing ArgoCD Prometheus metrics for CFR measurement? - Does ArgoCD already export `argocd_app_sync_total` on this cluster? - If yes, can Grafana query it directly without a new collector? - If no, what is the minimal webhook/notification setup required? ``` 2. **Deliverables do not match template requirements.** The template mandates two specific outputs: - A `docs/{topic}.md` file (the durable artifact, merged via docs-only PR) - Follow-up tickets created or updated with refined scope The current deliverables list a "decision document posted as a comment on this issue" -- this contradicts the template. Spike knowledge must live in a `docs/` file, not an issue comment. Comments are ephemeral; docs files are durable. 3. **Time-box is missing entirely.** Every spike must have a maximum time and an escalation clause ("if time-box expires without answer: close spike, document findings, escalate to Lucas for direction"). ### BLOCKERS 1. **Missing `### Time-box` section** -- Required by spike template. Without a time-box, spikes become open-ended investigations. Must include maximum time (e.g., "2 hours" or "1 session") and escalation clause. 2. **Deliverables do not include `docs/` file** -- Template requires `docs/{topic}.md` as a durable artifact. "Decision document posted as a comment" does not satisfy this. Change to: `docs/argocd-cfr-integration.md` (or similar). 3. **Deliverables do not include follow-up tickets** -- Template requires "Follow-up tickets created or existing tickets updated with refined scope." This is the whole point of a spike: investigation produces scoped work items. 4. **Section headers do not match template** -- `### Investigation Questions` should be `### Question`; `### Spike Deliverables` should be `### Deliverables`. Standardized headers enable tooling and agent parsing. ### NITS - Remove `### User Story` -- spikes do not have user stories per template guidance. The context section already covers the "why." - `### Context` and `### Constraints` are not in the template but contain useful information. Consider folding their content into the `### Question` section as background context, or keep them as supplementary sections below the required ones. - The `### Related` section uses `dora-metrics` as the project slug -- verify this matches the actual pal-e-docs project slug. ### VERDICT: NOT APPROVED The conversion from Feature to Spike was the right call -- the investigation framing is much better. However, three required spike template sections are missing or non-conformant: `### Time-box` (absent), `### Deliverables` (wrong outputs), and `### Question` (wrong header and format). These are structural blockers per template conformance rules. Fix the section headers, add a time-box with escalation clause, and align deliverables to the two required spike outputs (`docs/` file + follow-up tickets).
ldraney commented 2026-06-13 19:25:00 +00:00
Author
Owner
Copy link

Updated per re-review: added time-box, fixed section headers to match template-issue-spike, changed deliverable from issue comment to docs/ artifact, added follow-up tickets as required output.

Updated per re-review: added time-box, fixed section headers to match template-issue-spike, changed deliverable from issue comment to docs/ artifact, added follow-up tickets as required output.
ldraney commented 2026-06-13 19:30:34 +00:00
Author
Owner
Copy link

Issue #6 Template Review (Re-review #2)

TEMPLATE CONFORMANCE

  • ### Type present and valid (Spike)
  • ### Lineage present and non-empty
  • ### Repo present and non-empty
  • ### Question present and non-empty (previously ### Investigation Questions -- FIXED)
  • ### Deliverables present and non-empty (previously ### Spike Deliverables -- FIXED)
  • ### Time-box present and non-empty (previously MISSING -- FIXED)
  • ### Related present and non-empty

Extra sections present (not in template, not a problem): ### User Story, ### Context, ### Constraints. These add useful context.

PREVIOUS BLOCKER RESOLUTION

All six previously identified blockers are resolved:

# Previous Blocker Status
1 Missing ### Time-box section RESOLVED -- "2 hours max" with escalation clause
2 Non-standard ### Investigation Questions header RESOLVED -- now ### Question
3 Non-standard ### Spike Deliverables header RESOLVED -- now ### Deliverables
4 Deliverables lacked durable docs/ artifact RESOLVED -- docs/argocd-cfr-decision.md listed
5 Deliverables lacked follow-up ticket creation RESOLVED -- follow-up ticket creation included
6 #7 cross-reference missing or unclear RESOLVED -- present in ### Related with clear dependency rationale

CONTENT QUALITY

Question section: Three numbered questions cover the investigation space well: (1) check if ArgoCD already exports metrics, (2) evaluate direct query path, (3) scope the webhook fallback. The decision tree is clear.

Deliverables: Both required spike outputs are present with checkboxes:

  • docs/argocd-cfr-decision.md -- durable artifact with specified content (rationale, tradeoffs, config changes)
  • Follow-up ticket creation with "if no action needed, the docs file explains why" escape clause

Time-box: "2 hours max" with a specific escalation path: "escalate to Lucas for a decision on whether to pursue webhook approach without full investigation." This is concrete and actionable.

Related: #7 cross-reference includes dependency rationale ("deployment event source determines MTTR measurement approach"). Project slug dora-metrics is listed.

NITS

  1. Question format: Template recommends framing the top-level as a single yes/no or "which approach" question with sub-questions as bullets. Current format uses three parallel numbered questions. Consider restructuring as: "What is the right deployment-event source for CFR calculation -- existing ArgoCD Prometheus metrics, or a new webhook integration?" with the three current questions as sub-bullets. Minor style point.

  2. Deliverables checkbox text: The second deliverable could name the likely target repo (e.g., pal-e-dora-exporter or pal-e-platform) rather than "the appropriate repo." This is a minor specificity improvement, not a blocker since the spike's purpose is to determine exactly this.

VERDICT: APPROVED

## Issue #6 Template Review (Re-review #2) ### TEMPLATE CONFORMANCE - [x] `### Type` present and valid (Spike) - [x] `### Lineage` present and non-empty - [x] `### Repo` present and non-empty - [x] `### Question` present and non-empty (previously `### Investigation Questions` -- FIXED) - [x] `### Deliverables` present and non-empty (previously `### Spike Deliverables` -- FIXED) - [x] `### Time-box` present and non-empty (previously MISSING -- FIXED) - [x] `### Related` present and non-empty Extra sections present (not in template, not a problem): `### User Story`, `### Context`, `### Constraints`. These add useful context. ### PREVIOUS BLOCKER RESOLUTION All six previously identified blockers are resolved: | # | Previous Blocker | Status | |---|-----------------|--------| | 1 | Missing `### Time-box` section | RESOLVED -- "2 hours max" with escalation clause | | 2 | Non-standard `### Investigation Questions` header | RESOLVED -- now `### Question` | | 3 | Non-standard `### Spike Deliverables` header | RESOLVED -- now `### Deliverables` | | 4 | Deliverables lacked durable `docs/` artifact | RESOLVED -- `docs/argocd-cfr-decision.md` listed | | 5 | Deliverables lacked follow-up ticket creation | RESOLVED -- follow-up ticket creation included | | 6 | #7 cross-reference missing or unclear | RESOLVED -- present in `### Related` with clear dependency rationale | ### CONTENT QUALITY **Question section:** Three numbered questions cover the investigation space well: (1) check if ArgoCD already exports metrics, (2) evaluate direct query path, (3) scope the webhook fallback. The decision tree is clear. **Deliverables:** Both required spike outputs are present with checkboxes: - `docs/argocd-cfr-decision.md` -- durable artifact with specified content (rationale, tradeoffs, config changes) - Follow-up ticket creation with "if no action needed, the docs file explains why" escape clause **Time-box:** "2 hours max" with a specific escalation path: "escalate to Lucas for a decision on whether to pursue webhook approach without full investigation." This is concrete and actionable. **Related:** #7 cross-reference includes dependency rationale ("deployment event source determines MTTR measurement approach"). Project slug `dora-metrics` is listed. ### NITS 1. **Question format:** Template recommends framing the top-level as a single yes/no or "which approach" question with sub-questions as bullets. Current format uses three parallel numbered questions. Consider restructuring as: "What is the right deployment-event source for CFR calculation -- existing ArgoCD Prometheus metrics, or a new webhook integration?" with the three current questions as sub-bullets. Minor style point. 2. **Deliverables checkbox text:** The second deliverable could name the likely target repo (e.g., `pal-e-dora-exporter` or `pal-e-platform`) rather than "the appropriate repo." This is a minor specificity improvement, not a blocker since the spike's purpose is to determine exactly this. ### VERDICT: APPROVED
ldraney commented 2026-06-13 19:53:47 +00:00
Author
Owner
Copy link

Spike Complete -- ArgoCD CFR Investigation

Recommendation: Option A -- ServiceMonitor for ArgoCD native metrics

ArgoCD already emits argocd_app_sync_total with phase labels (Succeeded/Failed/Error) on port 8082. Adding a ServiceMonitor to scrape these metrics is the simplest path -- zero code changes to the exporter.

Key Findings

  1. ArgoCD is deployed at argocd-server.argocd.svc.cluster.local in the argocd namespace, but it is NOT managed by Terraform (no module exists). It appears as a blackbox probe target only.

  2. No ServiceMonitor exists for ArgoCD. Prometheus is not scraping ArgoCD's built-in metrics endpoint. The serviceMonitorSelectorNilUsesHelmValues = false setting means adding a ServiceMonitor will be auto-discovered.

  3. CFR is calculable in pure PromQL without any exporter changes:

    sum(increase(argocd_app_sync_total{phase=~"Failed|Error"}[7d])) by (name)
/ clamp_min(sum(increase(argocd_app_sync_total{phase=~"Succeeded|Failed|Error"}[7d])) by (name), 1)

  4. Three options evaluated (ServiceMonitor vs webhook vs API polling). ServiceMonitor wins on effort (~1hr vs 4-8hr), risk, and simplicity.

Prerequisite -- Manual Cluster Check

Before creating follow-up tickets, confirm metrics are emitted:

kubectl get svc -n argocd | grep metrics
kubectl run -i --rm curl-test --image=curlimages/curl -- \
  curl -s http://argocd-metrics.argocd.svc.cluster.local:8082/metrics | grep argocd_app_sync_total

Follow-up Tickets (pending cluster check)

  • pal-e-platform: Add ArgoCD ServiceMonitor for Prometheus scraping
  • pal-e-platform: Add ArgoCD CFR panel to DORA Grafana dashboard
  • pal-e-dora-exporter#7: MTTR calculation can now use ArgoCD sync events as the deployment event source

Deliverable

Full decision record: docs/argocd-cfr-decision.md -- PR #8

## Spike Complete -- ArgoCD CFR Investigation ### Recommendation: Option A -- ServiceMonitor for ArgoCD native metrics ArgoCD already emits `argocd_app_sync_total` with `phase` labels (Succeeded/Failed/Error) on port 8082. Adding a ServiceMonitor to scrape these metrics is the simplest path -- zero code changes to the exporter. ### Key Findings 1. **ArgoCD is deployed** at `argocd-server.argocd.svc.cluster.local` in the `argocd` namespace, but it is NOT managed by Terraform (no module exists). It appears as a blackbox probe target only. 2. **No ServiceMonitor exists for ArgoCD.** Prometheus is not scraping ArgoCD's built-in metrics endpoint. The `serviceMonitorSelectorNilUsesHelmValues = false` setting means adding a ServiceMonitor will be auto-discovered. 3. **CFR is calculable in pure PromQL** without any exporter changes: ```promql sum(increase(argocd_app_sync_total{phase=~"Failed|Error"}[7d])) by (name) / clamp_min(sum(increase(argocd_app_sync_total{phase=~"Succeeded|Failed|Error"}[7d])) by (name), 1) ``` 4. **Three options evaluated** (ServiceMonitor vs webhook vs API polling). ServiceMonitor wins on effort (~1hr vs 4-8hr), risk, and simplicity. ### Prerequisite -- Manual Cluster Check Before creating follow-up tickets, confirm metrics are emitted: ```bash kubectl get svc -n argocd | grep metrics kubectl run -i --rm curl-test --image=curlimages/curl -- \ curl -s http://argocd-metrics.argocd.svc.cluster.local:8082/metrics | grep argocd_app_sync_total ``` ### Follow-up Tickets (pending cluster check) - **pal-e-platform**: Add ArgoCD ServiceMonitor for Prometheus scraping - **pal-e-platform**: Add ArgoCD CFR panel to DORA Grafana dashboard - **pal-e-dora-exporter#7**: MTTR calculation can now use ArgoCD sync events as the deployment event source ### Deliverable Full decision record: `docs/argocd-cfr-decision.md` -- PR #8
ldraney 2026-06-13 19:55:36 +00:00
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
No results found.
Labels
Clear labels   
domain:backend

   
domain:devops

   
domain:frontend

   
status:approved

QA passed, awaiting merge approval

  
status:in-progress

Dev agent is actively working

  
status:needs-fix

QA found issues, back to dev

  
status:qa

PR submitted, awaiting QA review

  
type:bug

Bug fix

  
type:devops

Infrastructure/CI/config work

  
type:feature

New functionality

No labels
domain:backend
domain:devops
domain:frontend
status:approved
status:in-progress
status:needs-fix
status:qa
type:bug
type:devops
type:feature
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
ldraney/pal-e-dora-exporter#6
No description provided.