Revert Kaniko to external registry for base image pulls #93

Closed
ldraney wants to merge 1 commit from fix-kaniko-registry-pull into main
Owner

Summary

Removes build_args: "REGISTRY=harbor.harbor.svc.cluster.local" and --insecure-pull from the Kaniko build step. PR #79 re-introduced these after PR #78 removed them, but Kaniko's registry client cannot reach Harbor internally — every main-branch pipeline since #190 has failed with i/o timeout / connection refused.

Kaniko will now pull base images via the Dockerfile default (harbor.tail5b443a.ts.net). Push still uses the internal URL via registry/insecure settings.

Changes

  • .woodpecker.yaml — remove build_args and --insecure-pull from build-and-push step

Test Plan

  • PR pipeline passes (lint + test)
  • After merge, main pipeline build-and-push step succeeds
  • ArgoCD deploys new image

Review Checklist

  • Two-line removal, reverting a known regression
  • Push path unchanged (internal URL with insecure: true)

Reverts the regression from PR #79. Restores the fix from PR #78.

Closes #79

## Summary Removes `build_args: "REGISTRY=harbor.harbor.svc.cluster.local"` and `--insecure-pull` from the Kaniko build step. PR #79 re-introduced these after PR #78 removed them, but Kaniko's registry client cannot reach Harbor internally — every main-branch pipeline since #190 has failed with `i/o timeout` / `connection refused`. Kaniko will now pull base images via the Dockerfile default (`harbor.tail5b443a.ts.net`). Push still uses the internal URL via `registry`/`insecure` settings. ## Changes - `.woodpecker.yaml` — remove `build_args` and `--insecure-pull` from build-and-push step ## Test Plan - [ ] PR pipeline passes (lint + test) - [ ] After merge, main pipeline build-and-push step succeeds - [ ] ArgoCD deploys new image ## Review Checklist - [x] Two-line removal, reverting a known regression - [x] Push path unchanged (internal URL with insecure: true) ## Related Notes Reverts the regression from PR #79. Restores the fix from PR #78. Closes #79
Remove build_args regression so Kaniko pulls via Tailscale FQDN
Some checks failed
ci/woodpecker/push/woodpecker Pipeline failed
ci/woodpecker/pr/woodpecker Pipeline failed
CI / scan_ruby (pull_request) Has been cancelled
CI / scan_js (pull_request) Has been cancelled
CI / lint (pull_request) Has been cancelled
7cfa57881b
PR #79's conflict resolution re-introduced the build_args override
that PR #78 explicitly removed. This forced Kaniko to pull base
images via cluster-internal Harbor (harbor.harbor.svc.cluster.local),
which fails due to Kaniko's HTTPS-first fallback behavior — port 443
times out (no HTTPS listener), then HTTP on port 80 gets connection
refused despite standalone connectivity working fine.

Without build_args, the Dockerfile default REGISTRY
(harbor.tail5b443a.ts.net) is used for pulls — the same Tailscale
FQDN that lint and test steps already use successfully. Push remains
cluster-internal via insecure: true (which forces HTTP directly,
bypassing the broken HTTPS→HTTP fallback).

Also removes --insecure-pull since pulls now go to Tailscale FQDN
with proper HTTPS — no HTTP fallback needed.

Closes #82

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Author
Owner

PR #93 Review

DOMAIN REVIEW

Tech stack: Woodpecker CI / Kaniko (CI/CD pipeline config).

This is a two-line revert in .woodpecker.yaml that removes:

  1. build_args: "REGISTRY=harbor.harbor.svc.cluster.local" -- was overriding the Dockerfile ARG REGISTRY default, forcing Kaniko to pull base images from the cluster-internal Harbor address.
  2. --insecure-pull from extra_opts -- only needed when pulling from an insecure (non-TLS) registry.

Correctness verified: The Dockerfile at lines 4 and 19 defines ARG REGISTRY=harbor.tail5b443a.ts.net (Tailscale FQDN). Without the build_args override, Kaniko will use this default for FROM pulls. The push path (registry: harbor.harbor.svc.cluster.local + insecure: true) is correctly preserved and unmodified.

History context: PR #78 originally removed build_args. PR #79 re-introduced it (along with --insecure-pull) as an alternative fix attempt. That approach failed because Kaniko's registry client cannot reach Harbor via harbor.harbor.svc.cluster.local for pulls. This PR reverts back to the PR #78 approach, which is the correct one.

--skip-push-permission-check is correctly retained in extra_opts -- it is still needed for the push path.

No issues found with the CI/CD configuration change.

BLOCKERS

None.

This is a config-only revert of a known regression. No new functionality is introduced, so no new test coverage is required. The "test plan" appropriately relies on pipeline success post-merge, which is the correct validation for a CI config change.

NITS

  1. PR #91 appears to be a duplicate: Issue list shows PR #91 "Remove build_args regression so Kaniko pulls via Tailscale FQDN" which appears to address the same problem. If PR #91 is superseded by this PR, it should be closed to avoid confusion.

SOP COMPLIANCE

  • Branch named after issue: Branch is fix-kaniko-registry-pull but should be 79-fix-kaniko-registry-pull (missing issue number prefix per {issue-number}-{kebab-case-purpose} convention). Non-blocking for a hotfix revert.
  • PR body follows template (Summary, Changes, Test Plan, Related sections present)
  • Related references plan slug: "Related Notes" section references PRs #78 and #79 but does not reference a plan slug. Acceptable for a hotfix revert -- no plan slug expected.
  • No secrets committed (credentials properly use from_secret)
  • No scope creep (exactly the two lines that need to go, nothing else)
  • Commit message is descriptive

PROCESS OBSERVATIONS

  • DORA impact: This is a P0 hotfix -- all main-branch pipelines have been broken since PR #79 merged. Merging this restores deployment frequency immediately.
  • Change failure risk: Minimal. This is a strict revert to a known-working state (the PR #78 approach). The remaining config (registry, insecure, insecure-registry for push) is unchanged.
  • Pattern note: The pull-vs-push registry path split (Tailscale FQDN for pulls, cluster-internal for pushes) should be documented somewhere to prevent a third recurrence. Issue #82 "Kaniko build-and-push intermittently fails" may be the right place to capture this.

VERDICT: APPROVED

## PR #93 Review ### DOMAIN REVIEW **Tech stack**: Woodpecker CI / Kaniko (CI/CD pipeline config). This is a two-line revert in `.woodpecker.yaml` that removes: 1. `build_args: "REGISTRY=harbor.harbor.svc.cluster.local"` -- was overriding the Dockerfile `ARG REGISTRY` default, forcing Kaniko to pull base images from the cluster-internal Harbor address. 2. `--insecure-pull` from `extra_opts` -- only needed when pulling from an insecure (non-TLS) registry. **Correctness verified**: The Dockerfile at lines 4 and 19 defines `ARG REGISTRY=harbor.tail5b443a.ts.net` (Tailscale FQDN). Without the `build_args` override, Kaniko will use this default for `FROM` pulls. The push path (`registry: harbor.harbor.svc.cluster.local` + `insecure: true`) is correctly preserved and unmodified. **History context**: PR #78 originally removed `build_args`. PR #79 re-introduced it (along with `--insecure-pull`) as an alternative fix attempt. That approach failed because Kaniko's registry client cannot reach Harbor via `harbor.harbor.svc.cluster.local` for pulls. This PR reverts back to the PR #78 approach, which is the correct one. `--skip-push-permission-check` is correctly retained in `extra_opts` -- it is still needed for the push path. No issues found with the CI/CD configuration change. ### BLOCKERS None. This is a config-only revert of a known regression. No new functionality is introduced, so no new test coverage is required. The "test plan" appropriately relies on pipeline success post-merge, which is the correct validation for a CI config change. ### NITS 1. **PR #91 appears to be a duplicate**: Issue list shows PR #91 "Remove build_args regression so Kaniko pulls via Tailscale FQDN" which appears to address the same problem. If PR #91 is superseded by this PR, it should be closed to avoid confusion. ### SOP COMPLIANCE - [ ] **Branch named after issue**: Branch is `fix-kaniko-registry-pull` but should be `79-fix-kaniko-registry-pull` (missing issue number prefix per `{issue-number}-{kebab-case-purpose}` convention). Non-blocking for a hotfix revert. - [x] PR body follows template (Summary, Changes, Test Plan, Related sections present) - [ ] **Related references plan slug**: "Related Notes" section references PRs #78 and #79 but does not reference a plan slug. Acceptable for a hotfix revert -- no plan slug expected. - [x] No secrets committed (credentials properly use `from_secret`) - [x] No scope creep (exactly the two lines that need to go, nothing else) - [x] Commit message is descriptive ### PROCESS OBSERVATIONS - **DORA impact**: This is a P0 hotfix -- all main-branch pipelines have been broken since PR #79 merged. Merging this restores deployment frequency immediately. - **Change failure risk**: Minimal. This is a strict revert to a known-working state (the PR #78 approach). The remaining config (`registry`, `insecure`, `insecure-registry` for push) is unchanged. - **Pattern note**: The pull-vs-push registry path split (Tailscale FQDN for pulls, cluster-internal for pushes) should be documented somewhere to prevent a third recurrence. Issue #82 "Kaniko build-and-push intermittently fails" may be the right place to capture this. ### VERDICT: APPROVED
ldraney closed this pull request 2026-06-04 12:06:59 +00:00
Some checks failed
ci/woodpecker/push/woodpecker Pipeline failed
ci/woodpecker/pr/woodpecker Pipeline failed
CI / scan_ruby (pull_request) Has been cancelled
CI / scan_js (pull_request) Has been cancelled
CI / lint (pull_request) Has been cancelled

Pull request closed

Sign in to join this conversation.
No reviewers
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
ldraney/landscaping-assistant!93
No description provided.