Use cluster-internal registry with --insecure-pull for Kaniko #79

Merged

ldraney merged 1 commit from 77-kaniko-insecure-pull into main 2026-06-04 04:46:53 +00:00

Conversation 1 Commits 1 Files changed 1 +2 -1

ldraney commented 2026-06-04 04:43:56 +00:00

Owner

Copy link

Summary

• Supersedes PR #78 approach — Tailscale FQDN pulls also fail (DERP relay drops mid-transfer)
• Restores build_args for cluster-internal registry AND adds --insecure-pull to force HTTP for base image pulls
• This avoids both failure modes: no DERP relay dependency, no HTTPS timeout on port 443

Changes

• .woodpecker.yaml: Add build_args: "REGISTRY=harbor.harbor.svc.cluster.local" and --insecure-pull to extra_opts

Test Plan

• Tests pass locally
• Pipeline build-and-push succeeds — Kaniko pulls via HTTP from cluster-internal Harbor
• No regressions in lint, test, or image push

Review Checklist

• Passed automated review-fix loop
• No secrets committed
• No unnecessary file changes
• Commit messages are descriptive

Related Notes

• ldraney/landscaping-assistant #77 — Kaniko build-arg override breaks base image pull
• ldraney/landscaping-assistant #23 — parent CI optimization issue
• landscaping-assistant — project this work belongs to

Closes #77

## Summary - Supersedes PR #78 approach — Tailscale FQDN pulls also fail (DERP relay drops mid-transfer) - Restores `build_args` for cluster-internal registry AND adds `--insecure-pull` to force HTTP for base image pulls - This avoids both failure modes: no DERP relay dependency, no HTTPS timeout on port 443 ## Changes - `.woodpecker.yaml`: Add `build_args: "REGISTRY=harbor.harbor.svc.cluster.local"` and `--insecure-pull` to `extra_opts` ## Test Plan - [ ] Tests pass locally - [ ] Pipeline build-and-push succeeds — Kaniko pulls via HTTP from cluster-internal Harbor - [ ] No regressions in lint, test, or image push ## Review Checklist - [ ] Passed automated review-fix loop - [ ] No secrets committed - [ ] No unnecessary file changes - [ ] Commit messages are descriptive ## Related Notes - `ldraney/landscaping-assistant #77` — Kaniko build-arg override breaks base image pull - `ldraney/landscaping-assistant #23` — parent CI optimization issue - `landscaping-assistant` — project this work belongs to Closes #77

ldraney added 1 commit 2026-06-04 04:43:56 +00:00

Use cluster-internal registry with --insecure-pull for Kaniko ... 2b0ad5a81f

Both registry paths fail differently:
- Tailscale FQDN: DERP relay drops mid-layer-transfer
- Cluster-internal: Kaniko tries HTTPS first (times out on 443)

Adding --insecure-pull forces HTTP for base image pulls, avoiding
the HTTPS timeout. Combined with the build-arg override, this gives
a direct cluster path with no DERP relay dependency.

Closes #77

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

ldraney commented 2026-06-04 04:45:18 +00:00

Author

Owner

Copy link

PR #79 Review

DOMAIN REVIEW

Tech stack: Woodpecker CI pipeline (.woodpecker.yaml), Kaniko image builder, Dockerfile with multi-stage build using ARG REGISTRY for base image resolution.

Change analysis: The diff is a single-line modification in .woodpecker.yaml line 51:

-      extra_opts: "--skip-push-permission-check"
+      extra_opts: "--skip-push-permission-check --insecure-pull"

The build_args line (line 50) is already present on main per the current file -- the diff shows it as context, not a change. So the actual net change is adding --insecure-pull to extra_opts.

Correctness of the fix: The logic chain is sound:

1. build_args: "REGISTRY=harbor.harbor.svc.cluster.local" overrides the Dockerfile's ARG REGISTRY=harbor.tail5b443a.ts.net default, so FROM lines resolve to harbor.harbor.svc.cluster.local/library/ruby-rails-build:latest and ruby-rails-runtime:latest.
2. The insecure: true and insecure-registry settings handle the push side (Kaniko pushes to the registry over HTTP).
3. The problem was the pull side -- Kaniko's default behavior tries HTTPS:443 first for the FROM image pulls from the cluster-internal address. Harbor's cluster service only exposes HTTP:80, so HTTPS times out and HTTP gets refused on port 80 after the HTTPS attempt fails differently than expected.
4. --insecure-pull tells Kaniko to use HTTP for pulling base images. This correctly solves the pull-side protocol mismatch without touching the push path.

Kaniko flag review:

• --insecure-pull is a well-documented Kaniko flag that forces HTTP for image pulls. It is the correct flag for this scenario (distinct from --insecure which affects pushes, already handled by the plugin's insecure: true setting).
• The flag is additive and does not conflict with --skip-push-permission-check.

Security consideration: Using HTTP for base image pulls is acceptable here because the traffic stays within the Kubernetes cluster (pod-to-service communication on the cluster network). There is no external network traversal. The Tailscale FQDN alternative (harbor.tail5b443a.ts.net) would use HTTPS but routes through DERP relays which are unreliable for large layer transfers -- cluster-internal HTTP is the pragmatically correct choice.

BLOCKERS

None.

This is a CI configuration change, not application code. The BLOCKER criteria (test coverage for new functionality, unvalidated user input, secrets in code, DRY violations in auth paths) do not apply. No new application logic is introduced. Secrets continue to use from_secret references (lines 57-58).

NITS

1. Comment opportunity: A brief YAML comment above extra_opts explaining why --insecure-pull is needed would help future maintainers understand the cluster-internal HTTP requirement. Something like # --insecure-pull: cluster-internal Harbor serves HTTP only. Not blocking.

SOP COMPLIANCE

• Branch named after issue: 77-kaniko-insecure-pull follows {issue-number}-{kebab-case-purpose} convention
• PR body has Summary, Changes, Test Plan, Related sections
• Related section references issue #77 and parent issue #23
• Related section does not reference a plan slug from pal-e-docs (minor -- this is a hotfix-class change so a formal plan may not exist)
• No secrets committed -- credentials use from_secret references
• No unnecessary file changes -- single file, single line, tightly scoped
• Commit messages: PR title is descriptive and specific
• Closes #77 present in PR body

PROCESS OBSERVATIONS

• This is the third iteration on this CI registry problem (#73 -> #76, #77 -> #78, #77 -> #79). The progression shows good diagnostic discipline: PR #76 added the build-arg (fixed DERP for Dockerfile pulls), PR #78 tried removing it (hit DERP failures again), and this PR keeps the build-arg and adds --insecure-pull to handle the protocol mismatch. The fix addresses both failure modes identified in the issue.
• PR is marked mergeable: false -- this may indicate a merge conflict or branch state issue that should be resolved before merge.
• Change failure risk is low -- this is an additive flag on a CI step that only runs on main push events after lint and test pass.

VERDICT: APPROVED

## PR #79 Review ### DOMAIN REVIEW **Tech stack**: Woodpecker CI pipeline (`.woodpecker.yaml`), Kaniko image builder, Dockerfile with multi-stage build using `ARG REGISTRY` for base image resolution. **Change analysis**: The diff is a single-line modification in `.woodpecker.yaml` line 51: ``` - extra_opts: "--skip-push-permission-check" + extra_opts: "--skip-push-permission-check --insecure-pull" ``` The `build_args` line (line 50) is already present on `main` per the current file -- the diff shows it as context, not a change. So the actual net change is adding `--insecure-pull` to `extra_opts`. **Correctness of the fix**: The logic chain is sound: 1. `build_args: "REGISTRY=harbor.harbor.svc.cluster.local"` overrides the Dockerfile's `ARG REGISTRY=harbor.tail5b443a.ts.net` default, so `FROM` lines resolve to `harbor.harbor.svc.cluster.local/library/ruby-rails-build:latest` and `ruby-rails-runtime:latest`. 2. The `insecure: true` and `insecure-registry` settings handle the **push** side (Kaniko pushes to the registry over HTTP). 3. The problem was the **pull** side -- Kaniko's default behavior tries HTTPS:443 first for the `FROM` image pulls from the cluster-internal address. Harbor's cluster service only exposes HTTP:80, so HTTPS times out and HTTP gets refused on port 80 after the HTTPS attempt fails differently than expected. 4. `--insecure-pull` tells Kaniko to use HTTP for pulling base images. This correctly solves the pull-side protocol mismatch without touching the push path. **Kaniko flag review**: - `--insecure-pull` is a well-documented Kaniko flag that forces HTTP for image pulls. It is the correct flag for this scenario (distinct from `--insecure` which affects pushes, already handled by the plugin's `insecure: true` setting). - The flag is additive and does not conflict with `--skip-push-permission-check`. **Security consideration**: Using HTTP for base image pulls is acceptable here because the traffic stays within the Kubernetes cluster (pod-to-service communication on the cluster network). There is no external network traversal. The Tailscale FQDN alternative (`harbor.tail5b443a.ts.net`) would use HTTPS but routes through DERP relays which are unreliable for large layer transfers -- cluster-internal HTTP is the pragmatically correct choice. ### BLOCKERS None. This is a CI configuration change, not application code. The BLOCKER criteria (test coverage for new functionality, unvalidated user input, secrets in code, DRY violations in auth paths) do not apply. No new application logic is introduced. Secrets continue to use `from_secret` references (lines 57-58). ### NITS 1. **Comment opportunity**: A brief YAML comment above `extra_opts` explaining why `--insecure-pull` is needed would help future maintainers understand the cluster-internal HTTP requirement. Something like `# --insecure-pull: cluster-internal Harbor serves HTTP only`. Not blocking. ### SOP COMPLIANCE - [x] Branch named after issue: `77-kaniko-insecure-pull` follows `{issue-number}-{kebab-case-purpose}` convention - [x] PR body has Summary, Changes, Test Plan, Related sections - [x] Related section references issue #77 and parent issue #23 - [ ] Related section does not reference a plan slug from pal-e-docs (minor -- this is a hotfix-class change so a formal plan may not exist) - [x] No secrets committed -- credentials use `from_secret` references - [x] No unnecessary file changes -- single file, single line, tightly scoped - [x] Commit messages: PR title is descriptive and specific - [x] `Closes #77` present in PR body ### PROCESS OBSERVATIONS - This is the third iteration on this CI registry problem (#73 -> #76, #77 -> #78, #77 -> #79). The progression shows good diagnostic discipline: PR #76 added the build-arg (fixed DERP for Dockerfile pulls), PR #78 tried removing it (hit DERP failures again), and this PR keeps the build-arg and adds `--insecure-pull` to handle the protocol mismatch. The fix addresses both failure modes identified in the issue. - PR is marked `mergeable: false` -- this may indicate a merge conflict or branch state issue that should be resolved before merge. - Change failure risk is low -- this is an additive flag on a CI step that only runs on `main` push events after lint and test pass. ### VERDICT: APPROVED

ldraney force-pushed 77-kaniko-insecure-pull from 2b0ad5a81f to f15f8f7af8 2026-06-04 04:45:43 +00:00 Compare

ldraney merged commit 11cc138989 into main 2026-06-04 04:46:53 +00:00

ldraney deleted branch 77-kaniko-insecure-pull 2026-06-04 04:46:53 +00:00

ldraney referenced this pull request from a commit 2026-06-04 04:46:55 +00:00

Use cluster-internal registry with --insecure-pull for Kaniko (#79)

ldraney referenced this pull request 2026-06-04 05:22:25 +00:00

Kaniko build-and-push intermittently fails: cluster-internal Harbor unreachable #82

ldraney referenced this pull request from a commit 2026-06-04 12:02:45 +00:00

Remove build_args regression so Kaniko pulls via Tailscale FQDN

ldraney referenced this pull request 2026-06-04 12:03:39 +00:00

Remove build_args regression so Kaniko pulls via Tailscale FQDN #91

ldraney referenced this pull request 2026-06-04 12:05:04 +00:00

Remove build_args regression so Kaniko pulls via Tailscale FQDN #91

ldraney referenced this pull request 2026-06-04 12:05:21 +00:00

Revert Kaniko to external registry for base image pulls #93

ldraney referenced this pull request 2026-06-04 12:06:18 +00:00

Revert Kaniko to external registry for base image pulls #93

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels

No items

No labels

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

ldraney/landscaping-assistant!79

No description provided.