Skip to content

[SURE-12048] Fleet fails to reconcile MetalLB due to Server-Side Apply conflict on CRD caBundle field #5826

Open

Description

@khushalchandak17

Is there an existing issue for this?

  • I have searched the existing issues

Current Behavior

When MetalLB is installed via a Fleet GitRepo, the next reconcile (Force Update, or any fleet.yaml change) can fail with a Server-Side Apply conflict on the CRD's conversion webhook caBundle field. The GitRepo/Bundle gets stuck in WaitApplied / Failed and does not recover on its own.

Apply failed with 1 conflict: conflict with "MetalLB" using apiextensions.k8s.io/v1:
.spec.conversion.webhook.clientConfig.caBundle

Expected Behavior

Reconcile should succeed without conflict.

Steps To Reproduce

  1. Install Rancher 2.15.1 on a single-node RKE2 cluster with cert-manager.
  2. Create a GitRepo pointing at a repo with a fleet.yaml deploying the MetalLB Helm chart:
defaultNamespace: metallb
helm:
  chart: metallb
  repo: https://metallb.github.io/metallb
  version: 0.14.3
  name: metallb
  releaseName: metallb
  1. Wait for the initial install to succeed.
  2. Trigger a reconcile (Force Update, or push a fleet.yaml change).
  3. Check fleet-agent logs:
kubectl logs -n cattle-fleet-local-system deploy/fleet-agent --tail=200 | grep "conflict occurred"

Reproduces most reliably when step 4 happens shortly after a fresh install. On an already-settled install it does not always reproduce on the first retry.

Environment

- Architecture: arm64
- Fleet Version: 0.16.1
- Rancher Version: v2.15.1
- Cluster:
    - Provider: RKE2
    - Options: single node
    - Kubernetes Version: v1.36.4+rke2r1

Logs

{"level":"error","msg":"Reconciler error","BundleDeployment":{"name":"metallbchart-metallb"},"error":"failed deploying bundle: conflict occurred while applying object /bgppeers.metallb.io apiextensions.k8s.io/v1, Kind=CustomResourceDefinition: Apply failed with 1 conflict: conflict with \"MetalLB\" using apiextensions.k8s.io/v1: .spec.conversion.webhook.clientConfig.caBundle"}

Anything else?

Possibly related to #5368
SURE-12048

Activity

  1. changed the title [-]Fleet fails to reconcile MetalLB due to Server-Side Apply conflict on CRD caBundle field[/-] [+][SURE-12048] Fleet fails to reconcile MetalLB due to Server-Side Apply conflict on CRD caBundle field[/+] on Sep 22, 2026
  2. added theissue type on Sep 22, 2026
  3. moved this from 🆕 New to To Triage in Fleeton Sep 22, 2026
  4. added this to the v2.15.4 milestone on Sep 23, 2026
  5. moved this from To Triage to 📋 Backlog in Fleeton Sep 23, 2026
  6. self-assigned this
    on Oct 6, 2026
  7. 0xavi0 commented on Oct 7, 2026

    @0xavi0
    Contributor

    The problem seems to be that the MetalLB helm chart is using a placeholder for the initial value of .spec.conversion.webhook.clientConfig.caBundle.

    For supporting this we should be adding a new feature in Fleet to let it know what fields we want to treat differently.
    I don't know why this initial placeholder is created by MetalLB, but I tested adding this to the Bundle and it did work:

    apiVersion: kustomize.config.k8s.io/v1beta1
    kind: Kustomization
    patches:
      - target:
          group: apiextensions.k8s.io
          version: v1
          kind: CustomResourceDefinition
          name: bgppeers.metallb.io
        patch: |-
          - op: remove
            path: /spec/conversion/webhook/clientConfig/caBundle
    

    The Bundle layout would be something like:

    metallb/
    ├── fleet.yaml
    └── kustomization.yaml
    

    That deletes caBundle so it's never initially owned by Fleet and later the controller can deploy it with no problems.

    Would that workaround be enough @khushalchandak17 ?

  8. khushalchandak17 commented on Oct 9, 2026

    @khushalchandak17
    ContributorAuthor

    Thanks @0xavi0 for investigating. I will validate this workaround with our team and share the results. If accepted, we will document it as the short-term workaround.
    For the longer term, could we consider a Fleet RFE to support fields managed by third-party controllers?
    I will update this issue once validation is complete.

  9. weyfonk commented on Oct 28, 2026

    @weyfonk
    Contributor

    With a workaround being available, we would lean towards documenting that workaround instead of developing a bespoke fix for this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

Projects

  • Status
    🏗 In progress

Relationships

None yet

Development

No branches or pull requests

Issue actions