Add system test infrastructure and critical path browser specs #54

Merged

ldraney merged 3 commits from 52-add-system-test-infrastructure-and-criti into main 2026-06-01 07:21:00 +00:00

Conversation 3 Commits 3 Files changed 10 +535 -7

ldraney commented 2026-06-01 07:11:00 +00:00

Owner

Copy link

Summary

• Adds Capybara + Cuprite system test infrastructure for headless Chrome browser testing
• Implements three critical-path system specs covering JS-driven flows that request specs cannot verify (Turbo Stream DOM updates, Stimulus controller wiring)
• Updates Woodpecker CI to install Chromium so system specs run headless in the pipeline

Changes

• Gemfile / Gemfile.lock: added capybara and cuprite gems to :test group
• spec/support/system_test_config.rb: new -- configures Cuprite driver with headless Chrome, no-sandbox for CI, CHROMIUM_PATH env var support
• spec/rails_helper.rb: uncommented auto-require for spec/support/**/*.rb so system test config loads automatically
• spec/system/work_queue_spec.rb: new -- tests adding property to queue via quick-add form and toggling completion state
• spec/system/uploads_spec.rb: new -- tests upload page rendering, Stimulus controller wiring, and existing upload display
• spec/system/properties_spec.rb: new -- tests active/inactive toggle via Turbo Stream on manage page (both directions)
• .woodpecker.yaml: installs chromium in CI test step, sets CHROMIUM_PATH env var
• spec/fixtures/files/test_image.jpg: test fixture for upload spec

Test Plan

• bundle exec rspec spec/system/ -- 7 examples, 0 failures
• bundle exec rspec -- 86 examples, 0 failures (all existing specs unaffected)
• CI pipeline installs Chromium in ruby:3.4.9-slim and runs system specs headless

Review Checklist

• No secrets committed
• No unnecessary file changes (no app/ code modified, no request/model specs touched)
• Commit messages are descriptive

Related Notes

• Closes #52
• landscaping-assistant -- project board

## Summary - Adds Capybara + Cuprite system test infrastructure for headless Chrome browser testing - Implements three critical-path system specs covering JS-driven flows that request specs cannot verify (Turbo Stream DOM updates, Stimulus controller wiring) - Updates Woodpecker CI to install Chromium so system specs run headless in the pipeline ## Changes - `Gemfile` / `Gemfile.lock`: added `capybara` and `cuprite` gems to `:test` group - `spec/support/system_test_config.rb`: new -- configures Cuprite driver with headless Chrome, no-sandbox for CI, CHROMIUM_PATH env var support - `spec/rails_helper.rb`: uncommented auto-require for `spec/support/**/*.rb` so system test config loads automatically - `spec/system/work_queue_spec.rb`: new -- tests adding property to queue via quick-add form and toggling completion state - `spec/system/uploads_spec.rb`: new -- tests upload page rendering, Stimulus controller wiring, and existing upload display - `spec/system/properties_spec.rb`: new -- tests active/inactive toggle via Turbo Stream on manage page (both directions) - `.woodpecker.yaml`: installs `chromium` in CI test step, sets `CHROMIUM_PATH` env var - `spec/fixtures/files/test_image.jpg`: test fixture for upload spec ## Test Plan - [x] `bundle exec rspec spec/system/` -- 7 examples, 0 failures - [x] `bundle exec rspec` -- 86 examples, 0 failures (all existing specs unaffected) - [ ] CI pipeline installs Chromium in `ruby:3.4.9-slim` and runs system specs headless ## Review Checklist - [x] No secrets committed - [x] No unnecessary file changes (no app/ code modified, no request/model specs touched) - [x] Commit messages are descriptive ## Related Notes - Closes #52 - `landscaping-assistant` -- project board

ldraney added 1 commit 2026-06-01 07:11:00 +00:00

Add system test infrastructure and critical path browser specs ... 96dcde8ee5

Set up Capybara + Cuprite for headless Chrome system testing.
Add system specs for three critical JS-driven user paths:
work queue add/complete, upload page rendering, and property
active/inactive toggle. Update Woodpecker CI to install Chromium.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

ldraney commented 2026-06-01 07:11:43 +00:00

Author

Owner

Copy link

QA Review

Scope: 9 files changed, +222/-7 lines. System test infrastructure + 3 critical-path browser specs.

Findings

No blocking issues found.

Positives:

• Cuprite over Selenium is the right call for slim CI images -- lighter dependency chain, no chromedriver version management
• rack_test as default driver with cuprite only for system specs preserves fast non-JS test execution
• Upload spec pragmatically documents the CDP multipart limitation rather than writing a flaky test -- good engineering judgment
• Properties toggle spec correctly accounts for the filter controller hiding inactive properties by default
• All DOM selectors verified against actual view templates -- fill_in "client_name", .btn-complete, #manage_property_N, .queue-item, .is-completed all match
• No application code modified, no existing specs touched -- clean test-only PR per issue constraints
• CI config correctly adds chromium package and CHROMIUM_PATH env var for the ruby:3.4.9-slim image

Minor observations (non-blocking):

• The driven_by :cuprite in the before hook is slightly redundant with Capybara.javascript_driver = :cuprite since system specs default to the JS driver. However, being explicit is defensible and matches Rails convention.
• No gem version pins on capybara/cuprite. Acceptable for test-only gems but worth noting if lockfile reproducibility becomes a concern later.

Test Results

Per PR body: 7 system specs pass, 86 total specs pass, 0 failures.

VERDICT: APPROVE

## QA Review **Scope:** 9 files changed, +222/-7 lines. System test infrastructure + 3 critical-path browser specs. ### Findings No blocking issues found. **Positives:** - Cuprite over Selenium is the right call for slim CI images -- lighter dependency chain, no chromedriver version management - `rack_test` as default driver with `cuprite` only for system specs preserves fast non-JS test execution - Upload spec pragmatically documents the CDP multipart limitation rather than writing a flaky test -- good engineering judgment - Properties toggle spec correctly accounts for the filter controller hiding inactive properties by default - All DOM selectors verified against actual view templates -- `fill_in "client_name"`, `.btn-complete`, `#manage_property_N`, `.queue-item`, `.is-completed` all match - No application code modified, no existing specs touched -- clean test-only PR per issue constraints - CI config correctly adds `chromium` package and `CHROMIUM_PATH` env var for the `ruby:3.4.9-slim` image **Minor observations (non-blocking):** - The `driven_by :cuprite` in the before hook is slightly redundant with `Capybara.javascript_driver = :cuprite` since system specs default to the JS driver. However, being explicit is defensible and matches Rails convention. - No gem version pins on capybara/cuprite. Acceptable for test-only gems but worth noting if lockfile reproducibility becomes a concern later. ### Test Results Per PR body: 7 system specs pass, 86 total specs pass, 0 failures. **VERDICT: APPROVE**

ldraney commented 2026-06-01 07:14:28 +00:00

Author

Owner

Copy link

PR #54 Review

DOMAIN REVIEW

Stack: Ruby on Rails 8.1 / RSpec / Capybara+Cuprite / Turbo Streams / Stimulus / Woodpecker CI

Infrastructure (Cuprite config -- spec/support/system_test_config.rb):

• Headless Chrome flags are correct and CI-appropriate: no-sandbox, disable-gpu, disable-dev-shm-usage.
• CHROMIUM_PATH env var support allows CI to override the browser binary path. Clean pattern.
• Timeouts (process_timeout: 15, timeout: 10) are reasonable for CI. Local dev gets the same defaults, which is fine.
• Capybara.default_driver = :rack_test preserves fast non-JS specs; only system specs use Cuprite via driven_by :cuprite in the before block. Correct separation.

Woodpecker CI (.woodpecker.yaml):

• chromium added to apt-get in ruby:3.4.9-slim. This is sufficient -- Cuprite uses Chrome DevTools Protocol directly, no separate chromedriver binary needed. Correct.
• CHROMIUM_PATH: /usr/bin/chromium is the standard location in Debian-based slim images. Correct.

Gem selection (Gemfile):

• capybara and cuprite in :test group only. No selenium-webdriver (correct -- not needed). Clean.

rails_helper.rb change:

• Minimal and correct. Replaced the commented-out autoload block with an active one-liner plus a shorter comment. The 4-line comment removal + 2-line replacement is clean. No other changes to the helper.

Selector verification against actual view templates:

All selectors in the specs were verified against the actual ERB templates:

Spec selector	Template file	Match
#work_queue_item_#{id}	_queue_item.html.erb:1 -- dom_id(item)	YES
.queue-item	_queue_item.html.erb:1	YES
.btn-complete	_queue_item.html.erb:12	YES
.is-completed	_queue_item.html.erb:1 (conditional)	YES
#manage_property_#{id}	manage.html.erb:27 -- dom_id(property, :manage)	YES
have_button("Active"/"Inactive")	manage.html.erb:41 + toggle_active.turbo_stream.erb	YES
"Show Inactive"	manage.html.erb:19	YES
.upload-list-item	uploads/index.html.erb:18	YES
[data-controller='upload']	uploads/index.html.erb:4	YES
input[data-action='change->upload#submit']	uploads/index.html.erb:11	YES
label.upload-btn	uploads/index.html.erb:8	YES
"No photos yet"	uploads/index.html.erb:33 (substring match)	YES

Spec quality:

• Work queue spec (2 examples): Tests quick-add form submission via Turbo Stream append and completion toggle via Turbo Stream replace. Both verify DOM changes post-stream. The quick-add spec fills client_name and clicks "Add", which exercises the WorkQueueItemsController#create path with params[:client_name]. Correct.
• Properties spec (2 examples): Tests both toggle directions (active->inactive, inactive->active). The reactivation test correctly clicks "Show Inactive" first to reveal the hidden property. Both verify button text changes via Turbo Stream and confirm database state with property.reload. Thorough.
• Uploads spec (3 examples): Tests empty state, Stimulus wiring verification, and existing upload display. The CDP limitation on multipart auto-submit is well-documented with a clear NOTE comment explaining why the actual upload flow is not end-to-end tested. The alternative approach (verify wiring + rendering) is pragmatic.

Test fixture:

• spec/fixtures/files/test_image.jpg exists and is a valid JPEG. Size is reasonable for a test fixture.

BLOCKERS

None.

NITS

1. use_transactional_fixtures = true with system specs: Rails system specs with a JS driver (Cuprite) run the server in a separate thread. Transactional fixtures can cause the server thread to not see records created in the test transaction. This typically works in Rails 8.x because system tests use driven_by which handles the threading, but if you ever see intermittent "record not found" failures in system specs, this is the first thing to check. The canonical fix is to add DatabaseCleaner with truncation strategy for system specs, but given that all 7 examples pass locally, this is not blocking -- just something to watch for.

2. Upload spec -- Upload.new + manual attach pattern: The displays existing uploads example manually constructs an Upload with Upload.new + photo.attach + save!. This is fine for a system spec, but a factory or shared helper would reduce duplication if more upload specs are added later. Not blocking for 1 usage.

3. Missing file_fixture_upload usage: The upload spec uses File.open(file_fixture(...)) directly. For consistency with Rails conventions, file_fixture_upload("test_image.jpg", "image/jpeg") from ActionDispatch::TestProcess::FixtureFile could be used instead. Minor style preference -- what's there works.

SOP COMPLIANCE

• Branch named after issue: 52-add-system-test-infrastructure-and-criti (truncated but matches issue #52)
• PR body follows template: Summary, Changes, Test Plan, Related sections all present
• Related references project: "Closes #52" + landscaping-assistant project board reference
• No secrets committed (verified -- no API keys, tokens, or .env files in diff)
• No unnecessary file changes (9 files, all directly related to system test infrastructure -- no app code modified)
• Commit messages: PR title is descriptive; file-level changes are clear

PROCESS OBSERVATIONS

• This is the second of two testing tickets (#51 request specs, #52 system specs). Clean separation of concerns between the two.
• The CI pipeline test plan checkbox is unchecked ("CI pipeline installs Chromium..."). This is expected -- the CI run happens post-merge or on PR push. Worth confirming the pipeline passes before merge.
• 7 new system specs covering 3 critical JS paths (Turbo Streams, Stimulus wiring, filter interactions). Good coverage for the JS-driven flows that request specs cannot verify.
• Change failure risk: LOW. No application code modified. Test infrastructure only. Worst case failure mode is CI chromium installation issues, which would be caught by the pipeline itself.

VERDICT: APPROVED

## PR #54 Review ### DOMAIN REVIEW **Stack:** Ruby on Rails 8.1 / RSpec / Capybara+Cuprite / Turbo Streams / Stimulus / Woodpecker CI **Infrastructure (Cuprite config -- `spec/support/system_test_config.rb`):** - Headless Chrome flags are correct and CI-appropriate: `no-sandbox`, `disable-gpu`, `disable-dev-shm-usage`. - `CHROMIUM_PATH` env var support allows CI to override the browser binary path. Clean pattern. - Timeouts (`process_timeout: 15`, `timeout: 10`) are reasonable for CI. Local dev gets the same defaults, which is fine. - `Capybara.default_driver = :rack_test` preserves fast non-JS specs; only system specs use Cuprite via `driven_by :cuprite` in the before block. Correct separation. **Woodpecker CI (`.woodpecker.yaml`):** - `chromium` added to apt-get in `ruby:3.4.9-slim`. This is sufficient -- Cuprite uses Chrome DevTools Protocol directly, no separate chromedriver binary needed. Correct. - `CHROMIUM_PATH: /usr/bin/chromium` is the standard location in Debian-based slim images. Correct. **Gem selection (`Gemfile`):** - `capybara` and `cuprite` in `:test` group only. No `selenium-webdriver` (correct -- not needed). Clean. **rails_helper.rb change:** - Minimal and correct. Replaced the commented-out autoload block with an active one-liner plus a shorter comment. The 4-line comment removal + 2-line replacement is clean. No other changes to the helper. **Selector verification against actual view templates:** All selectors in the specs were verified against the actual ERB templates: | Spec selector | Template file | Match | |---|---|---| | `#work_queue_item_#{id}` | `_queue_item.html.erb:1` -- `dom_id(item)` | YES | | `.queue-item` | `_queue_item.html.erb:1` | YES | | `.btn-complete` | `_queue_item.html.erb:12` | YES | | `.is-completed` | `_queue_item.html.erb:1` (conditional) | YES | | `#manage_property_#{id}` | `manage.html.erb:27` -- `dom_id(property, :manage)` | YES | | `have_button("Active"/"Inactive")` | `manage.html.erb:41` + `toggle_active.turbo_stream.erb` | YES | | `"Show Inactive"` | `manage.html.erb:19` | YES | | `.upload-list-item` | `uploads/index.html.erb:18` | YES | | `[data-controller='upload']` | `uploads/index.html.erb:4` | YES | | `input[data-action='change->upload#submit']` | `uploads/index.html.erb:11` | YES | | `label.upload-btn` | `uploads/index.html.erb:8` | YES | | `"No photos yet"` | `uploads/index.html.erb:33` (substring match) | YES | **Spec quality:** - **Work queue spec** (2 examples): Tests quick-add form submission via Turbo Stream append and completion toggle via Turbo Stream replace. Both verify DOM changes post-stream. The quick-add spec fills `client_name` and clicks "Add", which exercises the `WorkQueueItemsController#create` path with `params[:client_name]`. Correct. - **Properties spec** (2 examples): Tests both toggle directions (active->inactive, inactive->active). The reactivation test correctly clicks "Show Inactive" first to reveal the hidden property. Both verify button text changes via Turbo Stream and confirm database state with `property.reload`. Thorough. - **Uploads spec** (3 examples): Tests empty state, Stimulus wiring verification, and existing upload display. The CDP limitation on multipart auto-submit is well-documented with a clear NOTE comment explaining why the actual upload flow is not end-to-end tested. The alternative approach (verify wiring + rendering) is pragmatic. **Test fixture:** - `spec/fixtures/files/test_image.jpg` exists and is a valid JPEG. Size is reasonable for a test fixture. ### BLOCKERS None. ### NITS 1. **`use_transactional_fixtures = true` with system specs**: Rails system specs with a JS driver (Cuprite) run the server in a separate thread. Transactional fixtures can cause the server thread to not see records created in the test transaction. This typically works in Rails 8.x because system tests use `driven_by` which handles the threading, but if you ever see intermittent "record not found" failures in system specs, this is the first thing to check. The canonical fix is to add `DatabaseCleaner` with truncation strategy for system specs, but given that all 7 examples pass locally, this is not blocking -- just something to watch for. 2. **Upload spec -- `Upload.new` + manual attach pattern**: The `displays existing uploads` example manually constructs an Upload with `Upload.new` + `photo.attach` + `save!`. This is fine for a system spec, but a factory or shared helper would reduce duplication if more upload specs are added later. Not blocking for 1 usage. 3. **Missing `file_fixture_upload` usage**: The upload spec uses `File.open(file_fixture(...))` directly. For consistency with Rails conventions, `file_fixture_upload("test_image.jpg", "image/jpeg")` from `ActionDispatch::TestProcess::FixtureFile` could be used instead. Minor style preference -- what's there works. ### SOP COMPLIANCE - [x] Branch named after issue: `52-add-system-test-infrastructure-and-criti` (truncated but matches issue #52) - [x] PR body follows template: Summary, Changes, Test Plan, Related sections all present - [x] Related references project: "Closes #52" + `landscaping-assistant` project board reference - [x] No secrets committed (verified -- no API keys, tokens, or .env files in diff) - [x] No unnecessary file changes (9 files, all directly related to system test infrastructure -- no app code modified) - [x] Commit messages: PR title is descriptive; file-level changes are clear ### PROCESS OBSERVATIONS - This is the second of two testing tickets (#51 request specs, #52 system specs). Clean separation of concerns between the two. - The CI pipeline test plan checkbox is unchecked ("CI pipeline installs Chromium..."). This is expected -- the CI run happens post-merge or on PR push. Worth confirming the pipeline passes before merge. - 7 new system specs covering 3 critical JS paths (Turbo Streams, Stimulus wiring, filter interactions). Good coverage for the JS-driven flows that request specs cannot verify. - Change failure risk: LOW. No application code modified. Test infrastructure only. Worst case failure mode is CI chromium installation issues, which would be caught by the pipeline itself. ### VERDICT: APPROVED

forgejo_admin added 1 commit 2026-06-01 07:15:27 +00:00

Add testing strategy doc with system test infrastructure details ... cf9e71a631

Documents the full test pyramid: 76 examples across model, request,
and system specs. Covers Cuprite/CDP architecture, CI pipeline config,
known CDP limitations, and remaining gaps.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

ldraney commented 2026-06-01 07:18:02 +00:00

Author

Owner

Copy link

Accuracy Audit: docs/testing-strategy.md

Every factual claim in the document was checked against the actual code in the worktree. Findings below.

---

Test Inventory: Example Counts

Model Specs -- Doc claims 18, actual is 19

Model	Doc claims	Actual it blocks	Verdict
Property	7	8 (valid without address_line, validates client_name, multiple services, .active returns only active, .active includes default, #maps_url coordinate, #maps_url address-only, #maps_url nil)	INACCURATE -- 8 not 7
Service	2	2	ACCURATE
PropertyService	1	1	ACCURATE
Upload	5	5	ACCURATE
WorkQueueItem	3	3	ACCURATE
Total	18	19	INACCURATE -- off by 1

The Property model spec has 8 examples. The doc's description ("Validations, services association, .active scope, #maps_url") is directionally correct but the count is wrong.

Request Specs -- Doc claims 51, actual is 60

Controller	Doc claims	Actual it blocks	Verdict
Properties	26	28 (manage: 2, toggle_active: 4, show: 6, edit: 3, update: 5, resolve: 8)	INACCURATE -- 28 not 26
WorkQueueItems	10	8 (index: 3, create: 1, update: 1, destroy: 1, reorder: 2)	INACCURATE -- 8 not 10
Weeks	10	13 (index: 9, move: 4)	INACCURATE -- 13 not 10
Uploads	7	8 (index: 2, create: 2, show: 1, edit: 1, update: 1, destroy: 1)	INACCURATE -- 8 not 7
Health	1	1	ACCURATE
Client Errors	2	2	ACCURATE
Total	51	60	INACCURATE -- off by 9

The doc's sub-descriptions of endpoints covered are largely correct (the right routes are listed), but every per-controller count except Health and Client Errors is wrong. The Weeks doc says "index (7 scenarios), move (4 scenarios)" but actual is index: 9, move: 4. The doc says WorkQueueItems has "index, create, update, destroy, reorder" which matches the endpoint coverage, but claims 10 examples when there are 8.

System Specs -- Doc claims 7, actual is 7

Feature	Doc claims	Actual it blocks	Verdict
Work Queue	2	2	ACCURATE
Properties	2	2	ACCURATE
Uploads	3	3	ACCURATE
Total	7	7	ACCURATE

Grand Total -- Doc claims 76, actual is 86

Doc: 18 + 51 + 7 = 76. Actual: 19 + 60 + 7 = 86. Off by 10.

---

Endpoints Covered Claims

Verified each request spec file against the doc's "Endpoints covered" column.

Spec file	Doc claim	Verdict
Properties: "manage, toggle_active, show, edit, update (HTML + JSON), resolve (8 scenarios)"	Actual has: manage (2), toggle_active (4 -- includes Turbo Stream), show (6 -- includes address, client, active status, notes, services), edit (3 -- includes services checkboxes), update (5 -- HTML redirect, service assignments, validation failure, JSON success, JSON errors), resolve (8). All described endpoints present.	ACCURATE (endpoints correct, counts wrong)
WorkQueueItems: "index, create, update, destroy, reorder"	Actual: index (3 -- 200, invalid date, inactive filtering), create (1), update/toggle (1), destroy (1), reorder (2 -- bulk + single). All endpoints present.	ACCURATE
Weeks: "index (7 scenarios), move (4 scenarios)"	Actual: index has 9 scenarios (200, active shown, inactive excluded, completion checkmarks, completion count, prev week nav, next week nav, invalid param, day-picker buttons). Move has 4.	INACCURATE -- index has 9 scenarios, not 7
Uploads: "index, create, show, edit, update, destroy"	All 6 CRUD endpoints present. Index has 2 examples (200 + lists existing), create has 2 (success + failure).	ACCURATE
Health: "GET /up"	Confirmed.	ACCURATE
Client Errors: "valid payload, invalid JSON"	Confirmed: 204 for valid, 422 for invalid JSON.	ACCURATE

---

Gemfile Claims

Claim	Verdict
gem "capybara" in test group	ACCURATE -- Gemfile line 63
gem "cuprite" in test group	ACCURATE -- Gemfile line 64
Test group only (not dev)	ACCURATE -- lines 62-65 are group :test do

Doc does not mention specific versions, so no version mismatch to flag.

---

.woodpecker.yaml Claims

Claim	Verdict
Installs chromium via apt-get alongside libpq-dev and build-essential	ACCURATE -- line 29: apt-get install -y libpq-dev build-essential libyaml-dev chromium (note: also installs libyaml-dev, not mentioned in doc -- minor omission)
Sets CHROMIUM_PATH: /usr/bin/chromium	ACCURATE -- line 37
All specs run in single bundle exec rspec	ACCURATE -- line 33
RuboCop runs with --fail-level=error	ACCURATE -- line 21
Kaniko builds on main branch only	ACCURATE -- lines 61-62: event: push, branch: main
Postgres 16 service container	ACCURATE -- line 67: image: postgres:16
Docker build blocks merge: "No (only runs on push to main)"	ACCURATE -- build step has when: event: push, branch: main and depends_on: [lint, test]

---

Cuprite Config (spec/support/system_test_config.rb)

Claim	Verdict
Headless Chrome flags: no-sandbox, disable-gpu, disable-dev-shm-usage	ACCURATE -- lines 15-17
Respects CHROMIUM_PATH env var	ACCURATE -- line 20
Default driver: rack_test	ACCURATE -- line 25
System specs use Cuprite automatically	ACCURATE -- lines 28-30: config.before(:each, type: :system) { driven_by :cuprite }

---

Remaining Gaps / Not Planned

Claim	Verdict
"No server-side NominatimService exists"	ACCURATE -- no app/services/ directory exists at all. Geocoding is entirely client-side JS.
"No server-side HTTP calls to external APIs" (VCR/WebMock not needed)	ACCURATE -- no external HTTP calls found server-side.
"Factory Bot overkill for current model count"	ACCURATE -- no FactoryBot in Gemfile; specs use Model.create! directly.
GPS/Geolocation not tested	ACCURATE -- no geolocation specs found.
Drag-to-reorder UI not tested	ACCURATE -- reorder is tested at request level only (PATCH /today/reorder).
Upload auto-submit not tested (CDP limitation)	ACCURATE -- upload system spec verifies Stimulus wiring only, not actual form submission.
Post-deploy smoke test not yet wired (#21)	Cannot verify issue existence, but no smoke test code found in repo. ACCURATE for the "not yet" claim.

---

System Spec CSS Selectors

Selector in doc	Actual usage in spec files	Verdict
.queue-item	work_queue_spec.rb line 18: have_css(".queue-item", text: ...) AND _queue_item.html.erb line 1: class="queue-item"	ACCURATE
.is-completed	work_queue_spec.rb lines 37, 41: have_css(".is-completed") AND _queue_item.html.erb line 1: 'is-completed' if item.completed?	ACCURATE
.btn-complete	work_queue_spec.rb line 38: find(".btn-complete").click AND _queue_item.html.erb line 12: class: "btn-icon btn-complete"	ACCURATE
#manage_property_	properties_spec.rb lines 18, 24, 42, 48: within("#manage_property_#{property.id}") AND manage.html.erb line 27: id="<%= dom_id(property, :manage) %>" (generates manage_property_N)	ACCURATE
#work_queue_item_	work_queue_spec.rb lines 36, 41: within("#work_queue_item_#{queue_item.id}") AND _queue_item.html.erb line 1: id="<%= dom_id(item) %>" (generates work_queue_item_N)	ACCURATE
data-controller='upload'	uploads_spec.rb line 25: have_css("[data-controller='upload']") AND uploads/index.html.erb line 4: data: { controller: "upload" }	ACCURATE
data-action='change->upload#submit'	uploads_spec.rb line 28: have_css("input[data-action='change->upload#submit']") AND uploads/index.html.erb line 11: data: { action: "change->upload#submit" }	ACCURATE

---

Critical Path Coverage Tables

The per-step coverage descriptions in the "Critical User Paths" section are accurate. Each "Request spec" or "System spec" claim maps to a real spec. Each "Not tested" claim was confirmed (no matching spec exists). The resolve endpoint scenarios listed (creates property, finds existing, assigns services, TBD default, preserves existing name) all have corresponding it blocks.

One note: Path 2 mentions "Resolve defaults: Request spec: TBD default, preserves existing name" -- this maps to 3 actual it blocks (TBD when no client_name, TBD when no street and no client_name, preserves existing name), not the 2 implied by the description.

---

Summary of Inaccuracies

All inaccuracies are count mismatches in the Test Inventory section. No structural or coverage claims are wrong.

Item	Doc says	Actual	Delta
Property model examples	7	8	+1
Model spec total	18	19	+1
Properties request examples	26	28	+2
WorkQueueItems request examples	10	8	-2
Weeks request examples	10	13	+3
Weeks index scenarios	7	9	+2
Uploads request examples	7	8	+1
Request spec total	51	60	+9
Grand total	76	86	+10

Everything else -- endpoint coverage, gem config, CI pipeline, Cuprite setup, CSS selectors, remaining gaps, not-planned rationale -- is accurate.

## Accuracy Audit: `docs/testing-strategy.md` Every factual claim in the document was checked against the actual code in the worktree. Findings below. --- ### Test Inventory: Example Counts #### Model Specs -- Doc claims 18, actual is 19 | Model | Doc claims | Actual `it` blocks | Verdict | |-------|-----------|-------------------|---------| | Property | 7 | **8** (valid without address_line, validates client_name, multiple services, .active returns only active, .active includes default, #maps_url coordinate, #maps_url address-only, #maps_url nil) | **INACCURATE** -- 8 not 7 | | Service | 2 | 2 | ACCURATE | | PropertyService | 1 | 1 | ACCURATE | | Upload | 5 | 5 | ACCURATE | | WorkQueueItem | 3 | 3 | ACCURATE | | **Total** | **18** | **19** | **INACCURATE** -- off by 1 | The Property model spec has 8 examples. The doc's description ("Validations, services association, `.active` scope, `#maps_url`") is directionally correct but the count is wrong. #### Request Specs -- Doc claims 51, actual is 60 | Controller | Doc claims | Actual `it` blocks | Verdict | |-----------|-----------|-------------------|---------| | Properties | 26 | **28** (manage: 2, toggle_active: 4, show: 6, edit: 3, update: 5, resolve: 8) | **INACCURATE** -- 28 not 26 | | WorkQueueItems | 10 | **8** (index: 3, create: 1, update: 1, destroy: 1, reorder: 2) | **INACCURATE** -- 8 not 10 | | Weeks | 10 | **13** (index: 9, move: 4) | **INACCURATE** -- 13 not 10 | | Uploads | 7 | **8** (index: 2, create: 2, show: 1, edit: 1, update: 1, destroy: 1) | **INACCURATE** -- 8 not 7 | | Health | 1 | 1 | ACCURATE | | Client Errors | 2 | 2 | ACCURATE | | **Total** | **51** | **60** | **INACCURATE** -- off by 9 | The doc's sub-descriptions of endpoints covered are largely correct (the right routes are listed), but every per-controller count except Health and Client Errors is wrong. The Weeks doc says "index (7 scenarios), move (4 scenarios)" but actual is index: 9, move: 4. The doc says WorkQueueItems has "index, create, update, destroy, reorder" which matches the endpoint coverage, but claims 10 examples when there are 8. #### System Specs -- Doc claims 7, actual is 7 | Feature | Doc claims | Actual `it` blocks | Verdict | |---------|-----------|-------------------|---------| | Work Queue | 2 | 2 | ACCURATE | | Properties | 2 | 2 | ACCURATE | | Uploads | 3 | 3 | ACCURATE | | **Total** | **7** | **7** | ACCURATE | #### Grand Total -- Doc claims 76, actual is 86 Doc: 18 + 51 + 7 = 76. Actual: 19 + 60 + 7 = **86**. Off by 10. --- ### Endpoints Covered Claims Verified each request spec file against the doc's "Endpoints covered" column. | Spec file | Doc claim | Verdict | |-----------|-----------|---------| | Properties: "manage, toggle_active, show, edit, update (HTML + JSON), resolve (8 scenarios)" | Actual has: manage (2), toggle_active (4 -- includes Turbo Stream), show (6 -- includes address, client, active status, notes, services), edit (3 -- includes services checkboxes), update (5 -- HTML redirect, service assignments, validation failure, JSON success, JSON errors), resolve (8). All described endpoints present. | ACCURATE (endpoints correct, counts wrong) | | WorkQueueItems: "index, create, update, destroy, reorder" | Actual: index (3 -- 200, invalid date, inactive filtering), create (1), update/toggle (1), destroy (1), reorder (2 -- bulk + single). All endpoints present. | ACCURATE | | Weeks: "index (7 scenarios), move (4 scenarios)" | Actual: index has **9** scenarios (200, active shown, inactive excluded, completion checkmarks, completion count, prev week nav, next week nav, invalid param, day-picker buttons). Move has 4. | **INACCURATE** -- index has 9 scenarios, not 7 | | Uploads: "index, create, show, edit, update, destroy" | All 6 CRUD endpoints present. Index has 2 examples (200 + lists existing), create has 2 (success + failure). | ACCURATE | | Health: "GET /up" | Confirmed. | ACCURATE | | Client Errors: "valid payload, invalid JSON" | Confirmed: 204 for valid, 422 for invalid JSON. | ACCURATE | --- ### Gemfile Claims | Claim | Verdict | |-------|---------| | `gem "capybara"` in test group | ACCURATE -- Gemfile line 63 | | `gem "cuprite"` in test group | ACCURATE -- Gemfile line 64 | | Test group only (not dev) | ACCURATE -- lines 62-65 are `group :test do` | Doc does not mention specific versions, so no version mismatch to flag. --- ### `.woodpecker.yaml` Claims | Claim | Verdict | |-------|---------| | Installs `chromium` via `apt-get` alongside `libpq-dev` and `build-essential` | ACCURATE -- line 29: `apt-get install -y libpq-dev build-essential libyaml-dev chromium` (note: also installs `libyaml-dev`, not mentioned in doc -- minor omission) | | Sets `CHROMIUM_PATH: /usr/bin/chromium` | ACCURATE -- line 37 | | All specs run in single `bundle exec rspec` | ACCURATE -- line 33 | | RuboCop runs with `--fail-level=error` | ACCURATE -- line 21 | | Kaniko builds on main branch only | ACCURATE -- lines 61-62: `event: push, branch: main` | | Postgres 16 service container | ACCURATE -- line 67: `image: postgres:16` | | Docker build blocks merge: "No (only runs on push to main)" | ACCURATE -- build step has `when: event: push, branch: main` and `depends_on: [lint, test]` | --- ### Cuprite Config (`spec/support/system_test_config.rb`) | Claim | Verdict | |-------|---------| | Headless Chrome flags: `no-sandbox`, `disable-gpu`, `disable-dev-shm-usage` | ACCURATE -- lines 15-17 | | Respects `CHROMIUM_PATH` env var | ACCURATE -- line 20 | | Default driver: `rack_test` | ACCURATE -- line 25 | | System specs use Cuprite automatically | ACCURATE -- lines 28-30: `config.before(:each, type: :system) { driven_by :cuprite }` | --- ### Remaining Gaps / Not Planned | Claim | Verdict | |-------|---------| | "No server-side NominatimService exists" | ACCURATE -- no `app/services/` directory exists at all. Geocoding is entirely client-side JS. | | "No server-side HTTP calls to external APIs" (VCR/WebMock not needed) | ACCURATE -- no external HTTP calls found server-side. | | "Factory Bot overkill for current model count" | ACCURATE -- no FactoryBot in Gemfile; specs use `Model.create!` directly. | | GPS/Geolocation not tested | ACCURATE -- no geolocation specs found. | | Drag-to-reorder UI not tested | ACCURATE -- reorder is tested at request level only (PATCH /today/reorder). | | Upload auto-submit not tested (CDP limitation) | ACCURATE -- upload system spec verifies Stimulus wiring only, not actual form submission. | | Post-deploy smoke test not yet wired (#21) | Cannot verify issue existence, but no smoke test code found in repo. ACCURATE for the "not yet" claim. | --- ### System Spec CSS Selectors | Selector in doc | Actual usage in spec files | Verdict | |----------------|---------------------------|---------| | `.queue-item` | `work_queue_spec.rb` line 18: `have_css(".queue-item", text: ...)` AND `_queue_item.html.erb` line 1: `class="queue-item"` | ACCURATE | | `.is-completed` | `work_queue_spec.rb` lines 37, 41: `have_css(".is-completed")` AND `_queue_item.html.erb` line 1: `'is-completed' if item.completed?` | ACCURATE | | `.btn-complete` | `work_queue_spec.rb` line 38: `find(".btn-complete").click` AND `_queue_item.html.erb` line 12: `class: "btn-icon btn-complete"` | ACCURATE | | `#manage_property_` | `properties_spec.rb` lines 18, 24, 42, 48: `within("#manage_property_#{property.id}")` AND `manage.html.erb` line 27: `id="<%= dom_id(property, :manage) %>"` (generates `manage_property_N`) | ACCURATE | | `#work_queue_item_` | `work_queue_spec.rb` lines 36, 41: `within("#work_queue_item_#{queue_item.id}")` AND `_queue_item.html.erb` line 1: `id="<%= dom_id(item) %>"` (generates `work_queue_item_N`) | ACCURATE | | `data-controller='upload'` | `uploads_spec.rb` line 25: `have_css("[data-controller='upload']")` AND `uploads/index.html.erb` line 4: `data: { controller: "upload" }` | ACCURATE | | `data-action='change->upload#submit'` | `uploads_spec.rb` line 28: `have_css("input[data-action='change->upload#submit']")` AND `uploads/index.html.erb` line 11: `data: { action: "change->upload#submit" }` | ACCURATE | --- ### Critical Path Coverage Tables The per-step coverage descriptions in the "Critical User Paths" section are accurate. Each "Request spec" or "System spec" claim maps to a real spec. Each "Not tested" claim was confirmed (no matching spec exists). The resolve endpoint scenarios listed (creates property, finds existing, assigns services, TBD default, preserves existing name) all have corresponding `it` blocks. One note: Path 2 mentions "Resolve defaults: Request spec: TBD default, preserves existing name" -- this maps to 3 actual `it` blocks (TBD when no client_name, TBD when no street and no client_name, preserves existing name), not the 2 implied by the description. --- ### Summary of Inaccuracies All inaccuracies are **count mismatches** in the Test Inventory section. No structural or coverage claims are wrong. | Item | Doc says | Actual | Delta | |------|----------|--------|-------| | Property model examples | 7 | 8 | +1 | | Model spec total | 18 | 19 | +1 | | Properties request examples | 26 | 28 | +2 | | WorkQueueItems request examples | 10 | 8 | -2 | | Weeks request examples | 10 | 13 | +3 | | Weeks index scenarios | 7 | 9 | +2 | | Uploads request examples | 7 | 8 | +1 | | Request spec total | 51 | 60 | +9 | | **Grand total** | **76** | **86** | **+10** | Everything else -- endpoint coverage, gem config, CI pipeline, Cuprite setup, CSS selectors, remaining gaps, not-planned rationale -- is accurate.

forgejo_admin added 1 commit 2026-06-01 07:18:40 +00:00

Fix example counts in testing strategy doc ... 5d4adac32b

Accuracy audit found all qualitative claims correct but example
counts were off by 10. Corrected: models 19, request 60, system 7,
total 86. Added per-action breakdowns to request spec table.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

ldraney merged commit 21514c9941 into main 2026-06-01 07:21:00 +00:00

ldraney deleted branch 52-add-system-test-infrastructure-and-criti 2026-06-01 07:21:00 +00:00

ldraney referenced this pull request from a commit 2026-06-01 07:21:01 +00:00

Add system test infrastructure and critical path browser specs (#54)

ldraney referenced this pull request 2026-06-03 03:37:40 +00:00

Drop system tests: adopt Rails 8 endpoint-first testing philosophy #63

ldraney referenced this pull request from a commit 2026-06-03 03:38:11 +00:00

Drop system tests: adopt Rails 8 endpoint-first testing philosophy

ldraney referenced this pull request 2026-06-03 03:38:36 +00:00

Drop system tests: adopt Rails 8 endpoint-first testing #64

ldraney referenced this pull request 2026-06-03 03:40:38 +00:00

Drop system tests: adopt Rails 8 endpoint-first testing #64

ldraney referenced this pull request from a commit 2026-06-03 03:42:11 +00:00

Drop system tests: adopt Rails 8 endpoint-first testing philosophy

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels

No items

No labels

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

2 participants

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

ldraney/landscaping-assistant!54

No description provided.