### Test report for 4628.1.2+nightly-20260603-2100 / arm64
**Platforms tested** : azure
🟢 ok **bpf.ig**; Succeeded: azure (1)
🟢 ok **cl.basic**; Succeeded: azure (1)
🟢 ok **cl.cloudinit.basic**; Succeeded: azure (1)
🟢 ok **cl.cloudinit.multipart-mime**; Succeeded: azure (2); Failed: azure (1)
Diagnostic output for azure, run 1
```
L1: " Error: _harness.go:646: Cluster failed starting machines: creating public ip: PUT https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kola-cluster-image-67e9?72cd9c/providers/Microsoft.Network/publicIPAddresses/ip-a6ab826483"
L2: "--------------------------------------------------------------------------------"
L3: "RESPONSE 409: 409 Conflict"
L4: "ERROR CODE: ResourceGroupBeingDeleted"
L5: "--------------------------------------------------------------------------------"
L6: "{"
L7: "__error__: {"
L8: "__code__: __ResourceGroupBeingDeleted__,"
L9: "__message__: __The resource group _kola-cluster-image-67e972cd9c_ is in deprovisioning state and cannot perform this operation.__"
L10: "}"
L11: "}"
L12: "--------------------------------------------------------------------------------_"
L13: " "
```
🟢 ok **cl.cloudinit.script**; Succeeded: azure (2); Failed: azure (1)
Diagnostic output for azure, run 1
```
L1: " Error: _harness.go:646: Cluster failed starting machines: waiting for machine to become active: GET https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kola-?cluster-image-67e972cd9c/providers/Microsoft.Compute/virtualMachines/ci-4628.1.2-n-a80e1fe3c3"
L2: "--------------------------------------------------------------------------------"
L3: "RESPONSE 404: 404 Not Found"
L4: "ERROR CODE: NotFound"
L5: "--------------------------------------------------------------------------------"
L6: "{"
L7: "__error__: {"
L8: "__code__: __NotFound__,"
L9: "__message__: __The entity was not found in this Azure location.__"
L10: "}"
L11: "}"
L12: "--------------------------------------------------------------------------------_"
L13: " "
```
🟢 ok **cl.etcd-member.discovery # SKIP**; Succeeded: azure (1)
🟢 ok **cl.etcd-member.etcdctlv3**; Succeeded: azure (1)
🟢 ok **cl.etcd-member.v2-backup-restore # SKIP**; Succeeded: azure (1)
🟢 ok **cl.flannel.vxlan # SKIP**; Succeeded: azure (1)
🟢 ok **cl.ignition.kargs**; Succeeded: azure (2); Failed: azure (1)
Diagnostic output for azure, run 1
```
L1: " Error: _harness.go:646: Cluster failed starting machines: creating public ip: PUT https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kola-cluster-image-67e9?72cd9c/providers/Microsoft.Network/publicIPAddresses/ip-4472876608"
L2: "--------------------------------------------------------------------------------"
L3: "RESPONSE 409: 409 Conflict"
L4: "ERROR CODE: ResourceGroupBeingDeleted"
L5: "--------------------------------------------------------------------------------"
L6: "{"
L7: "__error__: {"
L8: "__code__: __ResourceGroupBeingDeleted__,"
L9: "__message__: __The resource group _kola-cluster-image-67e972cd9c_ is in deprovisioning state and cannot perform this operation.__"
L10: "}"
L11: "}"
L12: "--------------------------------------------------------------------------------_"
L13: " "
```
🟢 ok **cl.ignition.luks**; Succeeded: azure (1)
🟢 ok **cl.ignition.misc.empty**; Succeeded: azure (1)
🟢 ok **cl.ignition.symlink**; Succeeded: azure (1)
🟢 ok **cl.ignition.translation**; Succeeded: azure (1)
🟢 ok **cl.ignition.v1.btrfsroot**; Succeeded: azure (1)
🟢 ok **cl.ignition.v1.ext4root**; Succeeded: azure (2); Failed: azure (1)
Diagnostic output for azure, run 1
```
L1: " Error: _harness.go:646: Cluster failed starting machines: creating public ip: PUT https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kola-cluster-image-67e9?72cd9c/providers/Microsoft.Network/publicIPAddresses/ip-2657ad48b6"
L2: "--------------------------------------------------------------------------------"
L3: "RESPONSE 409: 409 Conflict"
L4: "ERROR CODE: ResourceGroupBeingDeleted"
L5: "--------------------------------------------------------------------------------"
L6: "{"
L7: "__error__: {"
L8: "__code__: __ResourceGroupBeingDeleted__,"
L9: "__message__: __The resource group _kola-cluster-image-67e972cd9c_ is in deprovisioning state and cannot perform this operation.__"
L10: "}"
L11: "}"
L12: "--------------------------------------------------------------------------------_"
L13: " "
```
🟢 ok **cl.ignition.v1.groups**; Succeeded: azure (2); Failed: azure (1)
Diagnostic output for azure, run 1
```
L1: " Error: _harness.go:646: Cluster failed starting machines: creating public ip: PUT https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kola-cluster-image-67e9?72cd9c/providers/Microsoft.Network/publicIPAddresses/ip-e6e004fa3c"
L2: "--------------------------------------------------------------------------------"
L3: "RESPONSE 409: 409 Conflict"
L4: "ERROR CODE: ResourceGroupBeingDeleted"
L5: "--------------------------------------------------------------------------------"
L6: "{"
L7: "__error__: {"
L8: "__code__: __ResourceGroupBeingDeleted__,"
L9: "__message__: __The resource group _kola-cluster-image-67e972cd9c_ is in deprovisioning state and cannot perform this operation.__"
L10: "}"
L11: "}"
L12: "--------------------------------------------------------------------------------_"
L13: " "
```
🟢 ok **cl.ignition.v1.noop**; Succeeded: azure (2); Failed: azure (1)
Diagnostic output for azure, run 1
```
L1: " Error: _harness.go:646: Cluster failed starting machines: creating public ip: PUT https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kola-cluster-image-67e9?72cd9c/providers/Microsoft.Network/publicIPAddresses/ip-162bde57f9"
L2: "--------------------------------------------------------------------------------"
L3: "RESPONSE 409: 409 Conflict"
L4: "ERROR CODE: ResourceGroupBeingDeleted"
L5: "--------------------------------------------------------------------------------"
L6: "{"
L7: "__error__: {"
L8: "__code__: __ResourceGroupBeingDeleted__,"
L9: "__message__: __The resource group _kola-cluster-image-67e972cd9c_ is in deprovisioning state and cannot perform this operation.__"
L10: "}"
L11: "}"
L12: "--------------------------------------------------------------------------------_"
L13: " "
```
🟢 ok **cl.ignition.v1.once**; Succeeded: azure (2); Failed: azure (1)
Diagnostic output for azure, run 1
```
L1: " Error: _harness.go:646: Cluster failed starting machines: creating public ip: PUT https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kola-cluster-image-67e9?72cd9c/providers/Microsoft.Network/publicIPAddresses/ip-e13b9991d9"
L2: "--------------------------------------------------------------------------------"
L3: "RESPONSE 409: 409 Conflict"
L4: "ERROR CODE: ResourceGroupBeingDeleted"
L5: "--------------------------------------------------------------------------------"
L6: "{"
L7: "__error__: {"
L8: "__code__: __ResourceGroupBeingDeleted__,"
L9: "__message__: __The resource group _kola-cluster-image-67e972cd9c_ is in deprovisioning state and cannot perform this operation.__"
L10: "}"
L11: "}"
L12: "--------------------------------------------------------------------------------_"
L13: " "
```
🟢 ok **cl.ignition.v1.users**; Succeeded: azure (2); Failed: azure (1)
Diagnostic output for azure, run 1
```
L1: " Error: _harness.go:646: Cluster failed starting machines: machine __ci-4628.1.2-n-ef3d1fb196__ failed basic checks: no /etc/os-release file: EOF:_"
L2: " "
```
🟢 ok **cl.ignition.v1.xfsroot**; Succeeded: azure (1)
🟢 ok **cl.ignition.v2.btrfsroot**; Succeeded: azure (1)
🟢 ok **cl.ignition.v2.ext4root**; Succeeded: azure (1)
🟢 ok **cl.ignition.v2.noop**; Succeeded: azure (1)
🟢 ok **cl.ignition.v2.users**; Succeeded: azure (2); Failed: azure (1)
Diagnostic output for azure, run 1
```
L1: " Error: _harness.go:646: Cluster failed starting machines: creating public ip: PUT https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kola-cluster-image-67e9?72cd9c/providers/Microsoft.Network/publicIPAddresses/ip-9473f766cc"
L2: "--------------------------------------------------------------------------------"
L3: "RESPONSE 409: 409 Conflict"
L4: "ERROR CODE: ResourceGroupBeingDeleted"
L5: "--------------------------------------------------------------------------------"
L6: "{"
L7: "__error__: {"
L8: "__code__: __ResourceGroupBeingDeleted__,"
L9: "__message__: __The resource group _kola-cluster-image-67e972cd9c_ is in deprovisioning state and cannot perform this operation.__"
L10: "}"
L11: "}"
L12: "--------------------------------------------------------------------------------_"
L13: " "
```
🟢 ok **cl.ignition.v2.xfsroot**; Succeeded: azure (2); Failed: azure (1)
Diagnostic output for azure, run 1
```
L1: " Error: _harness.go:646: Cluster failed starting machines: creating public ip: PUT https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kola-cluster-image-67e9?72cd9c/providers/Microsoft.Network/publicIPAddresses/ip-21188d88f7"
L2: "--------------------------------------------------------------------------------"
L3: "RESPONSE 409: 409 Conflict"
L4: "ERROR CODE: ResourceGroupBeingDeleted"
L5: "--------------------------------------------------------------------------------"
L6: "{"
L7: "__error__: {"
L8: "__code__: __ResourceGroupBeingDeleted__,"
L9: "__message__: __The resource group _kola-cluster-image-67e972cd9c_ is in deprovisioning state and cannot perform this operation.__"
L10: "}"
L11: "}"
L12: "--------------------------------------------------------------------------------_"
L13: " "
```
🟢 ok **cl.ignition.v2_1.ext4checkexisting**; Succeeded: azure (2); Failed: azure (1)
Diagnostic output for azure, run 1
```
L1: " Error: _filesystem.go:415: Couldn_t start machine: creating public ip: PUT https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kola-cluster-image-67e972cd9c/?providers/Microsoft.Network/publicIPAddresses/ip-4805935c1f"
L2: "--------------------------------------------------------------------------------"
L3: "RESPONSE 409: 409 Conflict"
L4: "ERROR CODE: ResourceGroupBeingDeleted"
L5: "--------------------------------------------------------------------------------"
L6: "{"
L7: "__error__: {"
L8: "__code__: __ResourceGroupBeingDeleted__,"
L9: "__message__: __The resource group _kola-cluster-image-67e972cd9c_ is in deprovisioning state and cannot perform this operation.__"
L10: "}"
L11: "}"
L12: "--------------------------------------------------------------------------------_"
L13: " "
```
🟢 ok **cl.ignition.v2_1.swap**; Succeeded: azure (2); Failed: azure (1)
Diagnostic output for azure, run 1
```
L1: " Error: _harness.go:646: Cluster failed starting machines: creating public ip: PUT https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kola-cluster-image-67e9?72cd9c/providers/Microsoft.Network/publicIPAddresses/ip-99ef560032"
L2: "--------------------------------------------------------------------------------"
L3: "RESPONSE 409: 409 Conflict"
L4: "ERROR CODE: ResourceGroupBeingDeleted"
L5: "--------------------------------------------------------------------------------"
L6: "{"
L7: "__error__: {"
L8: "__code__: __ResourceGroupBeingDeleted__,"
L9: "__message__: __The resource group _kola-cluster-image-67e972cd9c_ is in deprovisioning state and cannot perform this operation.__"
L10: "}"
L11: "}"
L12: "--------------------------------------------------------------------------------_"
L13: " "
```
🟢 ok **cl.ignition.v2_1.vfat**; Succeeded: azure (1)
🟢 ok **cl.internet**; Succeeded: azure (1)
🟢 ok **cl.locksmith.cluster # SKIP**; Succeeded: azure (1)
🟢 ok **cl.metadata.azure**; Succeeded: azure (2); Failed: azure (1)
Diagnostic output for azure, run 1
```
L1: " Error: _harness.go:646: Cluster failed starting machines: creating public ip: PUT https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kola-cluster-image-67e9?72cd9c/providers/Microsoft.Network/publicIPAddresses/ip-8317359f23"
L2: "--------------------------------------------------------------------------------"
L3: "RESPONSE 409: 409 Conflict"
L4: "ERROR CODE: ResourceGroupBeingDeleted"
L5: "--------------------------------------------------------------------------------"
L6: "{"
L7: "__error__: {"
L8: "__code__: __ResourceGroupBeingDeleted__,"
L9: "__message__: __The resource group _kola-cluster-image-67e972cd9c_ is in deprovisioning state and cannot perform this operation.__"
L10: "}"
L11: "}"
L12: "--------------------------------------------------------------------------------_"
L13: " "
```
🟢 ok **cl.network.initramfs.second-boot**; Succeeded: azure (1)
🟢 ok **cl.network.iptables**; Succeeded: azure (1)
🟢 ok **cl.network.nftables**; Succeeded: azure (1)
🟢 ok **cl.network.wireguard**; Succeeded: azure (2); Failed: azure (1)
Diagnostic output for azure, run 1
```
L1: " Error: _harness.go:646: Cluster failed starting machines: creating public ip: PUT https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kola-cluster-image-67e9?72cd9c/providers/Microsoft.Network/publicIPAddresses/ip-66de1d7e61"
L2: "--------------------------------------------------------------------------------"
L3: "RESPONSE 409: 409 Conflict"
L4: "ERROR CODE: ResourceGroupBeingDeleted"
L5: "--------------------------------------------------------------------------------"
L6: "{"
L7: "__error__: {"
L8: "__code__: __ResourceGroupBeingDeleted__,"
L9: "__message__: __The resource group _kola-cluster-image-67e972cd9c_ is in deprovisioning state and cannot perform this operation.__"
L10: "}"
L11: "}"
L12: "--------------------------------------------------------------------------------_"
L13: " "
```
🟢 ok **cl.osreset.ignition-rerun**; Succeeded: azure (1)
🟢 ok **cl.overlay.cleanup**; Succeeded: azure (1)
🟢 ok **cl.swap_activation**; Succeeded: azure (1)
🟢 ok **cl.toolbox.dnf-install**; Succeeded: azure (1)
🟢 ok **cl.update.badverity**; Succeeded: azure (2); Failed: azure (1)
Diagnostic output for azure, run 1
```
L1: " Error: _harness.go:646: Cluster failed starting machines: creating public ip: PUT https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kola-cluster-image-67e9?72cd9c/providers/Microsoft.Network/publicIPAddresses/ip-5ebdba782e"
L2: "--------------------------------------------------------------------------------"
L3: "RESPONSE 409: 409 Conflict"
L4: "ERROR CODE: ResourceGroupBeingDeleted"
L5: "--------------------------------------------------------------------------------"
L6: "{"
L7: "__error__: {"
L8: "__code__: __ResourceGroupBeingDeleted__,"
L9: "__message__: __The resource group _kola-cluster-image-67e972cd9c_ is in deprovisioning state and cannot perform this operation.__"
L10: "}"
L11: "}"
L12: "--------------------------------------------------------------------------------_"
L13: " "
```
🟢 ok **cl.update.reboot**; Succeeded: azure (1)
🟢 ok **cl.users.shells**; Succeeded: azure (1)
🟢 ok **cl.verity**; Succeeded: azure (1)
🟢 ok **coreos.auth.verify**; Succeeded: azure (1)
🟢 ok **coreos.ignition.groups**; Succeeded: azure (1)
🟢 ok **coreos.ignition.once**; Succeeded: azure (2); Failed: azure (1)
Diagnostic output for azure, run 1
```
L1: " Error: _harness.go:646: Cluster failed starting machines: creating public ip: PUT https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kola-cluster-image-67e9?72cd9c/providers/Microsoft.Network/publicIPAddresses/ip-18c6bac5f0"
L2: "--------------------------------------------------------------------------------"
L3: "RESPONSE 409: 409 Conflict"
L4: "ERROR CODE: ResourceGroupBeingDeleted"
L5: "--------------------------------------------------------------------------------"
L6: "{"
L7: "__error__: {"
L8: "__code__: __ResourceGroupBeingDeleted__,"
L9: "__message__: __The resource group _kola-cluster-image-67e972cd9c_ is in deprovisioning state and cannot perform this operation.__"
L10: "}"
L11: "}"
L12: "--------------------------------------------------------------------------------_"
L13: " "
```
🟢 ok **coreos.ignition.resource.local**; Succeeded: azure (2); Failed: azure (1)
Diagnostic output for azure, run 1
```
L1: " Error: _harness.go:646: Cluster failed starting machines: creating public ip: PUT https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kola-cluster-image-67e9?72cd9c/providers/Microsoft.Network/publicIPAddresses/ip-4d4be0d4d3"
L2: "--------------------------------------------------------------------------------"
L3: "RESPONSE 409: 409 Conflict"
L4: "ERROR CODE: ResourceGroupBeingDeleted"
L5: "--------------------------------------------------------------------------------"
L6: "{"
L7: "__error__: {"
L8: "__code__: __ResourceGroupBeingDeleted__,"
L9: "__message__: __The resource group _kola-cluster-image-67e972cd9c_ is in deprovisioning state and cannot perform this operation.__"
L10: "}"
L11: "}"
L12: "--------------------------------------------------------------------------------_"
L13: " "
```
🟢 ok **coreos.ignition.resource.remote**; Succeeded: azure (2); Failed: azure (1)
Diagnostic output for azure, run 1
```
L1: " Error: _harness.go:646: Cluster failed starting machines: creating public ip: PUT https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kola-cluster-image-67e9?72cd9c/providers/Microsoft.Network/publicIPAddresses/ip-9844489530"
L2: "--------------------------------------------------------------------------------"
L3: "RESPONSE 409: 409 Conflict"
L4: "ERROR CODE: ResourceGroupBeingDeleted"
L5: "--------------------------------------------------------------------------------"
L6: "{"
L7: "__error__: {"
L8: "__code__: __ResourceGroupBeingDeleted__,"
L9: "__message__: __The resource group _kola-cluster-image-67e972cd9c_ is in deprovisioning state and cannot perform this operation.__"
L10: "}"
L11: "}"
L12: "--------------------------------------------------------------------------------_"
L13: " "
```
🟢 ok **coreos.ignition.security.tls**; Succeeded: azure (1)
🟢 ok **coreos.ignition.ssh.key**; Succeeded: azure (1)
🟢 ok **coreos.ignition.systemd.enable-service**; Succeeded: azure (2); Failed: azure (1)
Diagnostic output for azure, run 1
```
L1: " Error: _harness.go:646: Cluster failed starting machines: creating public ip: PUT https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kola-cluster-image-67e9?72cd9c/providers/Microsoft.Network/publicIPAddresses/ip-9f498fdb65"
L2: "--------------------------------------------------------------------------------"
L3: "RESPONSE 409: 409 Conflict"
L4: "ERROR CODE: ResourceGroupBeingDeleted"
L5: "--------------------------------------------------------------------------------"
L6: "{"
L7: "__error__: {"
L8: "__code__: __ResourceGroupBeingDeleted__,"
L9: "__message__: __The resource group _kola-cluster-image-67e972cd9c_ is in deprovisioning state and cannot perform this operation.__"
L10: "}"
L11: "}"
L12: "--------------------------------------------------------------------------------_"
L13: " "
```
🟢 ok **coreos.locksmith.reboot**; Succeeded: azure (2); Failed: azure (1)
Diagnostic output for azure, run 1
```
L1: " Error: _locksmith.go:141: failed to check rebooted machine: ssh unreachable or system not ready: context deadline exceeded_"
L2: " "
```
🟢 ok **coreos.locksmith.tls**; Succeeded: azure (1)
🟢 ok **coreos.selinux.boolean**; Succeeded: azure (2); Failed: azure (1)
Diagnostic output for azure, run 1
```
L1: " Error: _harness.go:646: Cluster failed starting machines: creating public ip: PUT https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kola-cluster-image-67e9?72cd9c/providers/Microsoft.Network/publicIPAddresses/ip-2220c3b7f0"
L2: "--------------------------------------------------------------------------------"
L3: "RESPONSE 409: 409 Conflict"
L4: "ERROR CODE: ResourceGroupBeingDeleted"
L5: "--------------------------------------------------------------------------------"
L6: "{"
L7: "__error__: {"
L8: "__code__: __ResourceGroupBeingDeleted__,"
L9: "__message__: __The resource group _kola-cluster-image-67e972cd9c_ is in deprovisioning state and cannot perform this operation.__"
L10: "}"
L11: "}"
L12: "--------------------------------------------------------------------------------_"
L13: " "
```
🟢 ok **coreos.selinux.enforce**; Succeeded: azure (1)
🟢 ok **coreos.tls.fetch-urls**; Succeeded: azure (1)
🟢 ok **coreos.update.badusr**; Succeeded: azure (2); Failed: azure (1)
Diagnostic output for azure, run 1
```
L1: " Error: _update.go:168: ssh unreachable or system not ready: failure checking if machine is running: systemctl is-system-running returned stdout: ____, stderr: ____, err: ssh: handshake failed: ssh: un?able to authenticate, attempted methods [none publickey], no supported methods remain, systemctl list-jobs returned stdout: ____, stderr: ____, err: ssh: handshake failed: ssh: unable to authenticate,? attempted methods [none publickey], no supported methods remain_"
L2: " "
```
🟢 ok **docker.base**; Succeeded: azure (2); Failed: azure (1)
Diagnostic output for azure, run 1
```
L1: " Error: _harness.go:646: Cluster failed starting machines: creating public ip: PUT https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kola-cluster-image-67e9?72cd9c/providers/Microsoft.Network/publicIPAddresses/ip-dffe9a55ce"
L2: "--------------------------------------------------------------------------------"
L3: "RESPONSE 409: 409 Conflict"
L4: "ERROR CODE: ResourceGroupBeingDeleted"
L5: "--------------------------------------------------------------------------------"
L6: "{"
L7: "__error__: {"
L8: "__code__: __ResourceGroupBeingDeleted__,"
L9: "__message__: __The resource group _kola-cluster-image-67e972cd9c_ is in deprovisioning state and cannot perform this operation.__"
L10: "}"
L11: "}"
L12: "--------------------------------------------------------------------------------_"
L13: " "
```
🟢 ok **docker.btrfs-storage**; Succeeded: azure (2); Failed: azure (1)
Diagnostic output for azure, run 1
```
L1: " Error: _harness.go:646: Cluster failed starting machines: creating public ip: PUT https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kola-cluster-image-67e9?72cd9c/providers/Microsoft.Network/publicIPAddresses/ip-75a9da6ecc"
L2: "--------------------------------------------------------------------------------"
L3: "RESPONSE 409: 409 Conflict"
L4: "ERROR CODE: ResourceGroupBeingDeleted"
L5: "--------------------------------------------------------------------------------"
L6: "{"
L7: "__error__: {"
L8: "__code__: __ResourceGroupBeingDeleted__,"
L9: "__message__: __The resource group _kola-cluster-image-67e972cd9c_ is in deprovisioning state and cannot perform this operation.__"
L10: "}"
L11: "}"
L12: "--------------------------------------------------------------------------------_"
L13: " "
```
🟢 ok **docker.containerd-restart**; Succeeded: azure (2); Failed: azure (1)
Diagnostic output for azure, run 1
```
L1: " Error: _cluster.go:125: Unable to find image _ghcr.io/flatcar/busybox:latest_ locally"
L2: "cluster.go:125: latest: Pulling from flatcar/busybox"
L3: "cluster.go:125: dbc3eadfebd7: Pulling fs layer"
L4: "cluster.go:125: dbc3eadfebd7: Verifying Checksum"
L5: "cluster.go:125: dbc3eadfebd7: Download complete"
L6: "cluster.go:125: dbc3eadfebd7: Pull complete"
L7: "cluster.go:125: Digest: sha256:93e8234eb9ca92b9aae20fd73d6c9447ac3d1cc741c6e80c737f821dca582a0e"
L8: "cluster.go:125: Status: Downloaded newer image for ghcr.io/flatcar/busybox:latest"
L9: "cluster.go:145: __systemctl show containerd -p SubState --value__ failed: output , status ssh: unexpected packet in response to channel open:
🟢 ok **docker.enable-service.sysext**; Succeeded: azure (1)
🟢 ok **docker.lib-coreos-dockerd-compat**; Succeeded: azure (2); Failed: azure (1)
Diagnostic output for azure, run 1
```
L1: " _docker.lib-coreos-dockerd-compat/networks-reliably (49.72s)"
L2: "cluster.go:125: #0 building with __default__ instance using docker driver"
L3: "cluster.go:125: "
L4: "cluster.go:125: #1 [internal] load build definition from Dockerfile"
L5: "cluster.go:125: #1 transferring dockerfile: 108B done"
L6: "cluster.go:125: #1 DONE 0.0s"
L7: "cluster.go:125: "
L8: "cluster.go:125: #2 [internal] load .dockerignore"
L9: "cluster.go:125: #2 transferring context: 2B done"
L10: "cluster.go:125: #2 DONE 0.0s"
L11: "cluster.go:125: "
L12: "cluster.go:125: #3 [internal] load build context"
L13: "cluster.go:125: #3 transferring context: 4.09MB 0.0s done"
L14: "cluster.go:125: #3 DONE 0.1s"
L15: "cluster.go:125: "
L16: "cluster.go:125: #4 [1/1] COPY . /"
L17: "cluster.go:125: #4 DONE 0.0s"
L18: "cluster.go:125: "
L19: "cluster.go:125: #5 exporting to image"
L20: "cluster.go:125: #5 exporting layers 0.0s done"
L21: "cluster.go:125: #5 writing image sha256:dcb84b078c70a5a6833b7e794e95404382091e83c050c144c9f43db9322eafe9 done"
L22: "cluster.go:125: #5 naming to docker.io/library/ping 0.0s done"
L23: "cluster.go:125: #5 DONE 0.1s"
L24: "cluster.go:145: __for i in $(seq 1 100); do_n_t_techo -n ___$i: ____n_t_tdocker run --rm ping sh -c _ping -i 0.2 172.17.0.1 -w 1 _/dev/null && echo PASS || echo FAIL__n_tdone__ failed: output 1: PASS"
L25: "2: PASS"
L26: "3: PASS"
L27: "4: PASS"
L28: "5: PASS"
L29: "6: PASS"
L30: "7: PASS"
L31: "8: PASS"
L32: "9: PASS"
L33: "10: PASS"
L34: "11: PASS"
L35: "12: PASS"
L36: "13: PASS"
L37: "14: PASS"
L38: "15: PASS"
L39: "16: PASS"
L40: "17: PASS"
L41: "18: PASS"
L42: "19: PASS"
L43: "20: PASS"
L44: "21: PASS"
L45: "22: PASS"
L46: "23: PASS"
L47: "24: PASS"
L48: "25: PASS"
L49: "26: PASS"
L50: "27: PASS"
L51: "28: PASS"
L52: "29: PASS"
L53: "30: PASS"
L54: "31: PASS"
L55: "32: PASS"
L56: "33:, status wait: remote command exited without exit status or exit signal"
L57: "--- FAIL: docker.lib-coreos-dockerd-compat/user-no-caps (0.29s)"
L58: "cluster.go:145: __tmpdir=$(mktemp -d); cd $tmpdir; echo -e ___FROM scratch__nCOPY . /___ _ Dockerfile;_n_t b=$(which capsh sh grep cat ls); libs=$(sudo ldd $b | grep -o /lib_[^ ]*_ | sort -u);_?n_t sudo rsync -av --relative --copy-links $b $libs ./;_n_t sudo docker build -t captest .__ failed: output , status ssh: handshake failed: read tcp 10.200.1.4:52310-_20.94.202.255:22: r?ead: connection reset by peer"
L59: "--- FAIL: docker.lib-coreos-dockerd-compat/ownership (1.00s)"
L60: "cluster.go:145: __docker run --name ownership ghcr.io/flatcar/nginx stat -c ___%u/%g___ /etc/shadow__ failed: output , status dial tcp 20.94.202.255:22: connect: connection refused_"
L61: " "
```
🟢 ok **docker.network-openbsd-nc**; Succeeded: azure (2); Failed: azure (1)
Diagnostic output for azure, run 1
```
L1: " Error: _harness.go:646: Cluster failed starting machines: creating public ip: PUT https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kola-cluster-image-67e9?72cd9c/providers/Microsoft.Network/publicIPAddresses/ip-be2453f133"
L2: "--------------------------------------------------------------------------------"
L3: "RESPONSE 409: 409 Conflict"
L4: "ERROR CODE: ResourceGroupBeingDeleted"
L5: "--------------------------------------------------------------------------------"
L6: "{"
L7: "__error__: {"
L8: "__code__: __ResourceGroupBeingDeleted__,"
L9: "__message__: __The resource group _kola-cluster-image-67e972cd9c_ is in deprovisioning state and cannot perform this operation.__"
L10: "}"
L11: "}"
L12: "--------------------------------------------------------------------------------_"
L13: " "
```
🟢 ok **docker.selinux**; Succeeded: azure (1)
🟢 ok **docker.userns**; Succeeded: azure (1)
🟢 ok **kubeadm.v1.34.4.calico.base**; Succeeded: azure (3); Failed: azure (1, 2)
Diagnostic output for azure, run 2
```
L1: " Error: _cluster.go:125: I0604 00:36:26.830749 2755 version.go:260] remote version is much newer: v1.36.1; falling back to: stable-1.34"
L2: "cluster.go:125: [config/images] Pulled registry.k8s.io/kube-apiserver:v1.34.8"
L3: "cluster.go:125: [config/images] Pulled registry.k8s.io/kube-controller-manager:v1.34.8"
L4: "cluster.go:125: [config/images] Pulled registry.k8s.io/kube-scheduler:v1.34.8"
L5: "cluster.go:125: [config/images] Pulled registry.k8s.io/kube-proxy:v1.34.8"
L6: "cluster.go:125: [config/images] Pulled registry.k8s.io/coredns/coredns:v1.12.1"
L7: "cluster.go:125: [config/images] Pulled registry.k8s.io/pause:3.10.1"
L8: "cluster.go:125: [config/images] Pulled registry.k8s.io/etcd:3.6.5-0"
L9: "cluster.go:125: I0604 00:36:38.804654 3085 version.go:260] remote version is much newer: v1.36.1; falling back to: stable-1.34"
L10: "cluster.go:125: [init] Using Kubernetes version: v1.34.8"
L11: "cluster.go:125: [preflight] Running pre-flight checks"
L12: "cluster.go:125: [WARNING Hostname]: hostname __ci-4628.1.2-n-040fbb9c6c__ could not be reached"
L13: "cluster.go:125: [WARNING Hostname]: hostname __ci-4628.1.2-n-040fbb9c6c__: lookup ci-4628.1.2-n-040fbb9c6c on 168.63.129.16:53: no such host"
L14: "cluster.go:125: [WARNING Service-Kubelet]: kubelet service is not enabled, please run _systemctl enable kubelet.service_"
L15: "cluster.go:125: [preflight] Pulling images required for setting up a Kubernetes cluster"
L16: "cluster.go:125: [preflight] This might take a minute or two, depending on the speed of your internet connection"
L17: "cluster.go:125: [preflight] You can also perform this action beforehand using _kubeadm config images pull_"
L18: "cluster.go:125: [certs] Using certificateDir folder __/etc/kubernetes/pki__"
L19: "cluster.go:125: [certs] Generating __ca__ certificate and key"
L20: "cluster.go:125: [certs] Generating __apiserver__ certificate and key"
L21: "cluster.go:125: [certs] apiserver serving cert is signed for DNS names [ci-4628.1.2-n-040fbb9c6c kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.?96.0.1 10.0.0.13]"
L22: "cluster.go:125: [certs] Generating __apiserver-kubelet-client__ certificate and key"
L23: "cluster.go:125: [certs] Generating __front-proxy-ca__ certificate and key"
L24: "cluster.go:125: [certs] Generating __front-proxy-client__ certificate and key"
L25: "cluster.go:125: [certs] External etcd mode: Skipping etcd/ca certificate authority generation"
L26: "cluster.go:125: [certs] External etcd mode: Skipping etcd/server certificate generation"
L27: "cluster.go:125: [certs] External etcd mode: Skipping etcd/peer certificate generation"
L28: "cluster.go:125: [certs] External etcd mode: Skipping etcd/healthcheck-client certificate generation"
L29: "cluster.go:125: [certs] External etcd mode: Skipping apiserver-etcd-client certificate generation"
L30: "cluster.go:125: [certs] Generating __sa__ key and public key"
L31: "cluster.go:125: [kubeconfig] Using kubeconfig folder __/etc/kubernetes__"
L32: "cluster.go:125: [kubeconfig] Writing __admin.conf__ kubeconfig file"
L33: "cluster.go:125: [kubeconfig] Writing __super-admin.conf__ kubeconfig file"
L34: "cluster.go:125: [kubeconfig] Writing __kubelet.conf__ kubeconfig file"
L35: "cluster.go:125: [kubeconfig] Writing __controller-manager.conf__ kubeconfig file"
L36: "cluster.go:125: [kubeconfig] Writing __scheduler.conf__ kubeconfig file"
L37: "cluster.go:125: [control-plane] Using manifest folder __/etc/kubernetes/manifests__"
L38: "cluster.go:125: [control-plane] Creating static Pod manifest for __kube-apiserver__"
L39: "cluster.go:125: [control-plane] Creating static Pod manifest for __kube-controller-manager__"
L40: "cluster.go:125: [control-plane] Creating static Pod manifest for __kube-scheduler__"
L41: "cluster.go:125: [kubelet-start] Writing kubelet environment file with flags to file __/var/lib/kubelet/kubeadm-flags.env__"
L42: "cluster.go:125: [kubelet-start] Writing kubelet configuration to file __/var/lib/kubelet/instance-config.yaml__"
L43: "cluster.go:125: [patches] Applied patch of type __application/strategic-merge-patch+json__ to target __kubeletconfiguration__"
L44: "cluster.go:125: [kubelet-start] Writing kubelet configuration to file __/var/lib/kubelet/config.yaml__"
L45: "cluster.go:125: [kubelet-start] Starting the kubelet"
L46: "cluster.go:125: [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory __/etc/kubernetes/manifests__"
L47: "cluster.go:125: [kubelet-check] Waiting for a healthy kubelet at http://127.0.0.1:10248/healthz. This can take up to 4m0s"
L48: "cluster.go:125: [kubelet-check] The kubelet is healthy after 1.000787874s"
L49: "cluster.go:125: [control-plane-check] Waiting for healthy control plane components. This can take up to 30m0s"
L50: "cluster.go:125: [control-plane-check] Checking kube-apiserver at https://10.0.0.13:6443/livez"
L51: "cluster.go:125: [control-plane-check] Checking kube-controller-manager at https://127.0.0.1:10257/healthz"
L52: "cluster.go:125: [control-plane-check] Checking kube-scheduler at https://127.0.0.1:10259/livez"
L53: "cluster.go:125: [control-plane-check] kube-controller-manager is healthy after 1.505510496s"
L54: "cluster.go:125: [control-plane-check] kube-scheduler is healthy after 2.048973392s"
L55: "cluster.go:125: [control-plane-check] kube-apiserver is healthy after 3.500814765s"
L56: "cluster.go:125: [upload-config] Storing the configuration used in ConfigMap __kubeadm-config__ in the __kube-system__ Namespace"
L57: "cluster.go:125: [kubelet] Creating a ConfigMap __kubelet-config__ in namespace kube-system with the configuration for the kubelets in the cluster"
L58: "cluster.go:125: [upload-certs] Skipping phase. Please see --upload-certs"
L59: "cluster.go:125: [mark-control-plane] Marking the node ci-4628.1.2-n-040fbb9c6c as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-lo?ad-balancers]"
L60: "cluster.go:125: [mark-control-plane] Marking the node ci-4628.1.2-n-040fbb9c6c as control-plane by adding the taints [node-role.kubernetes.io/control-plane:NoSchedule]"
L61: "cluster.go:125: [bootstrap-token] Using token: 6fylvm.d51ud56t8m46qlbw"
L62: "cluster.go:125: [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles"
L63: "cluster.go:125: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes"
L64: "cluster.go:125: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials"
L65: "cluster.go:125: [bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token"
L66: "cluster.go:125: [bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster"
L67: "cluster.go:125: [bootstrap-token] Creating the __cluster-info__ ConfigMap in the __kube-public__ namespace"
L68: "cluster.go:125: [kubelet-finalize] Updating __/etc/kubernetes/kubelet.conf__ to point to a rotatable kubelet client certificate and key"
L69: "cluster.go:125: [addons] Applied essential addon: CoreDNS"
L70: "cluster.go:125: [addons] Applied essential addon: kube-proxy"
L71: "cluster.go:125: "
L72: "cluster.go:125: Your Kubernetes control-plane has initialized successfully!"
L73: "cluster.go:125: "
L74: "cluster.go:125: To start using your cluster, you need to run the following as a regular user:"
L75: "cluster.go:125: "
L76: "cluster.go:125: mkdir -p $HOME/.kube"
L77: "cluster.go:125: sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config"
L78: "cluster.go:125: sudo chown $(id -u):$(id -g) $HOME/.kube/config"
L79: "cluster.go:125: "
L80: "cluster.go:125: Alternatively, if you are the root user, you can run:"
L81: "cluster.go:125: "
L82: "cluster.go:125: export KUBECONFIG=/etc/kubernetes/admin.conf"
L83: "cluster.go:125: "
L84: "cluster.go:125: You should now deploy a pod network to the cluster."
L85: "cluster.go:125: Run __kubectl apply -f [podnetwork].yaml__ with one of the options listed at:"
L86: "cluster.go:125: https://kubernetes.io/docs/concepts/cluster-administration/addons/"
L87: "cluster.go:125: "
L88: "cluster.go:125: Then you can join any number of worker nodes by running the following on each as root:"
L89: "cluster.go:125: "
L90: "cluster.go:125: kubeadm join 10.0.0.13:6443 --token 6fylvm.d51ud56t8m46qlbw _"
L91: "cluster.go:125: --discovery-token-ca-cert-hash sha256:7ecdd407ef04ab8bc55b1c850eb6d2c32fdf87a62e1bc41bb9652c5a65612c8e "
L92: "cluster.go:125: namespace/tigera-operator created"
L93: "cluster.go:125: serviceaccount/tigera-operator created"
L94: "cluster.go:125: clusterrole.rbac.authorization.k8s.io/tigera-operator-secrets created"
L95: "cluster.go:125: clusterrole.rbac.authorization.k8s.io/tigera-operator created"
L96: "cluster.go:125: clusterrolebinding.rbac.authorization.k8s.io/tigera-operator created"
L97: "cluster.go:125: rolebinding.rbac.authorization.k8s.io/tigera-operator-secrets created"
L98: "cluster.go:125: deployment.apps/tigera-operator created"
L99: "cluster.go:125: customresourcedefinition.apiextensions.k8s.io/installations.operator.tigera.io condition met"
L100: "cluster.go:125: customresourcedefinition.apiextensions.k8s.io/installations.operator.tigera.io condition met"
L101: "cluster.go:125: customresourcedefinition.apiextensions.k8s.io/apiservers.operator.tigera.io condition met"
L102: "cluster.go:125: customresourcedefinition.apiextensions.k8s.io/apiservers.operator.tigera.io condition met"
L103: "cluster.go:125: installation.operator.tigera.io/default created"
L104: "cluster.go:125: apiserver.operator.tigera.io/default created"
L105: "cluster.go:125: goldmane.operator.tigera.io/default created"
L106: "cluster.go:125: whisker.operator.tigera.io/default created"
L107: "kubeadm.go:197: unable to setup cluster: unable to create worker node: waiting for machine to become active: GET https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourc?eGroups/kola-cluster-image-04d7414b6f/providers/Microsoft.Compute/virtualMachines/ci-4628.1.2-n-1eb36aac71"
L108: "--------------------------------------------------------------------------------"
L109: "RESPONSE 404: 404 Not Found"
L110: "ERROR CODE: NotFound"
L111: "--------------------------------------------------------------------------------"
L112: "{"
L113: "__error__: {"
L114: "__code__: __NotFound__,"
L115: "__message__: __The entity was not found in this Azure location.__"
L116: "}"
L117: "}"
L118: "--------------------------------------------------------------------------------_"
L119: " "
```
Diagnostic output for azure, run 1
```
L1: " Error: _cluster.go:125: I0603 23:36:51.066731 2808 version.go:260] remote version is much newer: v1.36.1; falling back to: stable-1.34"
L2: "cluster.go:125: [config/images] Pulled registry.k8s.io/kube-apiserver:v1.34.8"
L3: "cluster.go:125: [config/images] Pulled registry.k8s.io/kube-controller-manager:v1.34.8"
L4: "cluster.go:125: [config/images] Pulled registry.k8s.io/kube-scheduler:v1.34.8"
L5: "cluster.go:125: [config/images] Pulled registry.k8s.io/kube-proxy:v1.34.8"
L6: "cluster.go:125: [config/images] Pulled registry.k8s.io/coredns/coredns:v1.12.1"
L7: "cluster.go:125: [config/images] Pulled registry.k8s.io/pause:3.10.1"
L8: "cluster.go:125: [config/images] Pulled registry.k8s.io/etcd:3.6.5-0"
L9: "cluster.go:125: I0603 23:37:03.095716 3139 version.go:260] remote version is much newer: v1.36.1; falling back to: stable-1.34"
L10: "cluster.go:125: [init] Using Kubernetes version: v1.34.8"
L11: "cluster.go:125: [preflight] Running pre-flight checks"
L12: "cluster.go:125: [WARNING Hostname]: hostname __ci-4628.1.2-n-409b3fc056__ could not be reached"
L13: "cluster.go:125: [WARNING Hostname]: hostname __ci-4628.1.2-n-409b3fc056__: lookup ci-4628.1.2-n-409b3fc056 on 168.63.129.16:53: no such host"
L14: "cluster.go:125: [WARNING Service-Kubelet]: kubelet service is not enabled, please run _systemctl enable kubelet.service_"
L15: "cluster.go:125: [preflight] Pulling images required for setting up a Kubernetes cluster"
L16: "cluster.go:125: [preflight] This might take a minute or two, depending on the speed of your internet connection"
L17: "cluster.go:125: [preflight] You can also perform this action beforehand using _kubeadm config images pull_"
L18: "cluster.go:125: [certs] Using certificateDir folder __/etc/kubernetes/pki__"
L19: "cluster.go:125: [certs] Generating __ca__ certificate and key"
L20: "cluster.go:125: [certs] Generating __apiserver__ certificate and key"
L21: "cluster.go:125: [certs] apiserver serving cert is signed for DNS names [ci-4628.1.2-n-409b3fc056 kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.?96.0.1 10.0.0.17]"
L22: "cluster.go:125: [certs] Generating __apiserver-kubelet-client__ certificate and key"
L23: "cluster.go:125: [certs] Generating __front-proxy-ca__ certificate and key"
L24: "cluster.go:125: [certs] Generating __front-proxy-client__ certificate and key"
L25: "cluster.go:125: [certs] External etcd mode: Skipping etcd/ca certificate authority generation"
L26: "cluster.go:125: [certs] External etcd mode: Skipping etcd/server certificate generation"
L27: "cluster.go:125: [certs] External etcd mode: Skipping etcd/peer certificate generation"
L28: "cluster.go:125: [certs] External etcd mode: Skipping etcd/healthcheck-client certificate generation"
L29: "cluster.go:125: [certs] External etcd mode: Skipping apiserver-etcd-client certificate generation"
L30: "cluster.go:125: [certs] Generating __sa__ key and public key"
L31: "cluster.go:125: [kubeconfig] Using kubeconfig folder __/etc/kubernetes__"
L32: "cluster.go:125: [kubeconfig] Writing __admin.conf__ kubeconfig file"
L33: "cluster.go:125: [kubeconfig] Writing __super-admin.conf__ kubeconfig file"
L34: "cluster.go:125: [kubeconfig] Writing __kubelet.conf__ kubeconfig file"
L35: "cluster.go:125: [kubeconfig] Writing __controller-manager.conf__ kubeconfig file"
L36: "cluster.go:125: [kubeconfig] Writing __scheduler.conf__ kubeconfig file"
L37: "cluster.go:125: [control-plane] Using manifest folder __/etc/kubernetes/manifests__"
L38: "cluster.go:125: [control-plane] Creating static Pod manifest for __kube-apiserver__"
L39: "cluster.go:125: [control-plane] Creating static Pod manifest for __kube-controller-manager__"
L40: "cluster.go:125: [control-plane] Creating static Pod manifest for __kube-scheduler__"
L41: "cluster.go:125: [kubelet-start] Writing kubelet environment file with flags to file __/var/lib/kubelet/kubeadm-flags.env__"
L42: "cluster.go:125: [kubelet-start] Writing kubelet configuration to file __/var/lib/kubelet/instance-config.yaml__"
L43: "cluster.go:125: [patches] Applied patch of type __application/strategic-merge-patch+json__ to target __kubeletconfiguration__"
L44: "cluster.go:125: [kubelet-start] Writing kubelet configuration to file __/var/lib/kubelet/config.yaml__"
L45: "cluster.go:125: [kubelet-start] Starting the kubelet"
L46: "cluster.go:125: [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory __/etc/kubernetes/manifests__"
L47: "cluster.go:125: [kubelet-check] Waiting for a healthy kubelet at http://127.0.0.1:10248/healthz. This can take up to 4m0s"
L48: "cluster.go:125: [kubelet-check] The kubelet is healthy after 1.001141104s"
L49: "cluster.go:125: [control-plane-check] Waiting for healthy control plane components. This can take up to 30m0s"
L50: "cluster.go:125: [control-plane-check] Checking kube-apiserver at https://10.0.0.17:6443/livez"
L51: "cluster.go:125: [control-plane-check] Checking kube-controller-manager at https://127.0.0.1:10257/healthz"
L52: "cluster.go:125: [control-plane-check] Checking kube-scheduler at https://127.0.0.1:10259/livez"
L53: "cluster.go:125: [control-plane-check] kube-controller-manager is healthy after 1.505714006s"
L54: "cluster.go:125: [control-plane-check] kube-scheduler is healthy after 2.001083275s"
L55: "cluster.go:125: [control-plane-check] kube-apiserver is healthy after 5.001301826s"
L56: "cluster.go:125: [upload-config] Storing the configuration used in ConfigMap __kubeadm-config__ in the __kube-system__ Namespace"
L57: "cluster.go:125: [kubelet] Creating a ConfigMap __kubelet-config__ in namespace kube-system with the configuration for the kubelets in the cluster"
L58: "cluster.go:125: [upload-certs] Skipping phase. Please see --upload-certs"
L59: "cluster.go:125: [mark-control-plane] Marking the node ci-4628.1.2-n-409b3fc056 as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-lo?ad-balancers]"
L60: "cluster.go:125: [mark-control-plane] Marking the node ci-4628.1.2-n-409b3fc056 as control-plane by adding the taints [node-role.kubernetes.io/control-plane:NoSchedule]"
L61: "cluster.go:125: [bootstrap-token] Using token: h0qdqw.c0ko5m4kmm08pie8"
L62: "cluster.go:125: [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles"
L63: "cluster.go:125: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes"
L64: "cluster.go:125: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials"
L65: "cluster.go:125: [bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token"
L66: "cluster.go:125: [bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster"
L67: "cluster.go:125: [bootstrap-token] Creating the __cluster-info__ ConfigMap in the __kube-public__ namespace"
L68: "cluster.go:125: [kubelet-finalize] Updating __/etc/kubernetes/kubelet.conf__ to point to a rotatable kubelet client certificate and key"
L69: "cluster.go:125: [addons] Applied essential addon: CoreDNS"
L70: "cluster.go:125: [addons] Applied essential addon: kube-proxy"
L71: "cluster.go:125: "
L72: "cluster.go:125: Your Kubernetes control-plane has initialized successfully!"
L73: "cluster.go:125: "
L74: "cluster.go:125: To start using your cluster, you need to run the following as a regular user:"
L75: "cluster.go:125: "
L76: "cluster.go:125: mkdir -p $HOME/.kube"
L77: "cluster.go:125: sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config"
L78: "cluster.go:125: sudo chown $(id -u):$(id -g) $HOME/.kube/config"
L79: "cluster.go:125: "
L80: "cluster.go:125: Alternatively, if you are the root user, you can run:"
L81: "cluster.go:125: "
L82: "cluster.go:125: export KUBECONFIG=/etc/kubernetes/admin.conf"
L83: "cluster.go:125: "
L84: "cluster.go:125: You should now deploy a pod network to the cluster."
L85: "cluster.go:125: Run __kubectl apply -f [podnetwork].yaml__ with one of the options listed at:"
L86: "cluster.go:125: https://kubernetes.io/docs/concepts/cluster-administration/addons/"
L87: "cluster.go:125: "
L88: "cluster.go:125: Then you can join any number of worker nodes by running the following on each as root:"
L89: "cluster.go:125: "
L90: "cluster.go:125: kubeadm join 10.0.0.17:6443 --token h0qdqw.c0ko5m4kmm08pie8 _"
L91: "cluster.go:125: --discovery-token-ca-cert-hash sha256:8bad365ccd03e3539f35997cddbdde7f251b5d125e9f6e3f888512dcdab6a3c5 "
L92: "cluster.go:125: namespace/tigera-operator created"
L93: "cluster.go:125: serviceaccount/tigera-operator created"
L94: "cluster.go:125: clusterrole.rbac.authorization.k8s.io/tigera-operator-secrets created"
L95: "cluster.go:125: clusterrole.rbac.authorization.k8s.io/tigera-operator created"
L96: "cluster.go:125: clusterrolebinding.rbac.authorization.k8s.io/tigera-operator created"
L97: "cluster.go:125: rolebinding.rbac.authorization.k8s.io/tigera-operator-secrets created"
L98: "cluster.go:125: deployment.apps/tigera-operator created"
L99: "cluster.go:125: customresourcedefinition.apiextensions.k8s.io/installations.operator.tigera.io condition met"
L100: "cluster.go:125: customresourcedefinition.apiextensions.k8s.io/installations.operator.tigera.io condition met"
L101: "cluster.go:125: customresourcedefinition.apiextensions.k8s.io/apiservers.operator.tigera.io condition met"
L102: "cluster.go:125: customresourcedefinition.apiextensions.k8s.io/apiservers.operator.tigera.io condition met"
L103: "cluster.go:125: installation.operator.tigera.io/default created"
L104: "cluster.go:125: apiserver.operator.tigera.io/default created"
L105: "cluster.go:125: goldmane.operator.tigera.io/default created"
L106: "cluster.go:125: whisker.operator.tigera.io/default created"
L107: "cluster.go:125: W0603 23:39:00.517715 2588 joinconfiguration.go:112] [config] WARNING: Ignored configuration document with GroupVersionKind kubelet.config.k8s.io/v1beta1, Kind=KubeletConfiguration"
L108: "cluster.go:125: [WARNING Service-Kubelet]: kubelet service is not enabled, please run _systemctl enable kubelet.service_"
L109: "--- FAIL: kubeadm.v1.34.4.calico.base/node_readiness (21.38s)"
L110: "cluster.go:145: __kubectl get nodes | grep ___ Ready___| wc -l__ failed: output , status ssh: unexpected packet in response to channel open:
🟢 ok **kubeadm.v1.34.4.cilium.base**; Succeeded: azure (2); Failed: azure (1)
Diagnostic output for azure, run 1
```
L1: " Error: _kubeadm.go:197: unable to setup cluster: unable to create master node: creating public ip: PUT https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/ko?la-cluster-image-67e972cd9c/providers/Microsoft.Network/publicIPAddresses/ip-ec78ce68aa"
L2: "--------------------------------------------------------------------------------"
L3: "RESPONSE 409: 409 Conflict"
L4: "ERROR CODE: ResourceGroupBeingDeleted"
L5: "--------------------------------------------------------------------------------"
L6: "{"
L7: "__error__: {"
L8: "__code__: __ResourceGroupBeingDeleted__,"
L9: "__message__: __The resource group _kola-cluster-image-67e972cd9c_ is in deprovisioning state and cannot perform this operation.__"
L10: "}"
L11: "}"
L12: "--------------------------------------------------------------------------------_"
L13: " "
```
🟢 ok **kubeadm.v1.34.4.flannel.base**; Succeeded: azure (2); Failed: azure (1)
Diagnostic output for azure, run 1
```
L1: " Error: _kubeadm.go:197: unable to setup cluster: unable to create etcd node: creating public ip: PUT https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kola?-cluster-image-67e972cd9c/providers/Microsoft.Network/publicIPAddresses/ip-f79b57bfb6"
L2: "--------------------------------------------------------------------------------"
L3: "RESPONSE 409: 409 Conflict"
L4: "ERROR CODE: ResourceGroupBeingDeleted"
L5: "--------------------------------------------------------------------------------"
L6: "{"
L7: "__error__: {"
L8: "__code__: __ResourceGroupBeingDeleted__,"
L9: "__message__: __The resource group _kola-cluster-image-67e972cd9c_ is in deprovisioning state and cannot perform this operation.__"
L10: "}"
L11: "}"
L12: "--------------------------------------------------------------------------------_"
L13: " "
```
🟢 ok **kubeadm.v1.35.1.calico.base**; Succeeded: azure (2); Failed: azure (1)
Diagnostic output for azure, run 1
```
L1: " Error: _kubeadm.go:197: unable to setup cluster: unable to create etcd node: creating public ip: PUT https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kola?-cluster-image-67e972cd9c/providers/Microsoft.Network/publicIPAddresses/ip-3ca78b5503"
L2: "--------------------------------------------------------------------------------"
L3: "RESPONSE 409: 409 Conflict"
L4: "ERROR CODE: ResourceGroupBeingDeleted"
L5: "--------------------------------------------------------------------------------"
L6: "{"
L7: "__error__: {"
L8: "__code__: __ResourceGroupBeingDeleted__,"
L9: "__message__: __The resource group _kola-cluster-image-67e972cd9c_ is in deprovisioning state and cannot perform this operation.__"
L10: "}"
L11: "}"
L12: "--------------------------------------------------------------------------------_"
L13: " "
```
🟢 ok **kubeadm.v1.35.1.cilium.base**; Succeeded: azure (4); Failed: azure (1, 2, 3)
Diagnostic output for azure, run 3
```
L1: " Error: _cluster.go:125: I0604 00:48:00.602395 2762 version.go:260] remote version is much newer: v1.36.1; falling back to: stable-1.35"
L2: "cluster.go:125: [config/images] Pulled registry.k8s.io/kube-apiserver:v1.35.5"
L3: "cluster.go:125: [config/images] Pulled registry.k8s.io/kube-controller-manager:v1.35.5"
L4: "cluster.go:125: [config/images] Pulled registry.k8s.io/kube-scheduler:v1.35.5"
L5: "cluster.go:125: [config/images] Pulled registry.k8s.io/kube-proxy:v1.35.5"
L6: "cluster.go:125: [config/images] Pulled registry.k8s.io/coredns/coredns:v1.13.1"
L7: "cluster.go:125: [config/images] Pulled registry.k8s.io/pause:3.10.1"
L8: "cluster.go:125: [config/images] Pulled registry.k8s.io/etcd:3.6.6-0"
L9: "cluster.go:125: I0604 00:48:12.035620 3040 version.go:260] remote version is much newer: v1.36.1; falling back to: stable-1.35"
L10: "cluster.go:125: [init] Using Kubernetes version: v1.35.5"
L11: "cluster.go:125: [preflight] Running pre-flight checks"
L12: "cluster.go:125: [WARNING Hostname]: hostname __ci-4628.1.2-n-2bebd50bcd__ could not be reached"
L13: "cluster.go:125: [WARNING Hostname]: hostname __ci-4628.1.2-n-2bebd50bcd__: lookup ci-4628.1.2-n-2bebd50bcd on 168.63.129.16:53: no such host"
L14: "cluster.go:125: [WARNING Service-kubelet]: kubelet service is not enabled, please run _systemctl enable kubelet.service_"
L15: "cluster.go:125: [preflight] Pulling images required for setting up a Kubernetes cluster"
L16: "cluster.go:125: [preflight] This might take a minute or two, depending on the speed of your internet connection"
L17: "cluster.go:125: [preflight] You can also perform this action beforehand using _kubeadm config images pull_"
L18: "cluster.go:125: [certs] Using certificateDir folder __/etc/kubernetes/pki__"
L19: "cluster.go:125: [certs] Generating __ca__ certificate and key"
L20: "cluster.go:125: [certs] Generating __apiserver__ certificate and key"
L21: "cluster.go:125: [certs] apiserver serving cert is signed for DNS names [ci-4628.1.2-n-2bebd50bcd kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.?96.0.1 10.0.0.10]"
L22: "cluster.go:125: [certs] Generating __apiserver-kubelet-client__ certificate and key"
L23: "cluster.go:125: [certs] Generating __front-proxy-ca__ certificate and key"
L24: "cluster.go:125: [certs] Generating __front-proxy-client__ certificate and key"
L25: "cluster.go:125: [certs] External etcd mode: Skipping etcd/ca certificate authority generation"
L26: "cluster.go:125: [certs] External etcd mode: Skipping etcd/server certificate generation"
L27: "cluster.go:125: [certs] External etcd mode: Skipping etcd/peer certificate generation"
L28: "cluster.go:125: [certs] External etcd mode: Skipping etcd/healthcheck-client certificate generation"
L29: "cluster.go:125: [certs] External etcd mode: Skipping apiserver-etcd-client certificate generation"
L30: "cluster.go:125: [certs] Generating __sa__ key and public key"
L31: "cluster.go:125: [kubeconfig] Using kubeconfig folder __/etc/kubernetes__"
L32: "cluster.go:125: [kubeconfig] Writing __admin.conf__ kubeconfig file"
L33: "cluster.go:125: [kubeconfig] Writing __super-admin.conf__ kubeconfig file"
L34: "cluster.go:125: [kubeconfig] Writing __kubelet.conf__ kubeconfig file"
L35: "cluster.go:125: [kubeconfig] Writing __controller-manager.conf__ kubeconfig file"
L36: "cluster.go:125: [kubeconfig] Writing __scheduler.conf__ kubeconfig file"
L37: "cluster.go:125: [control-plane] Using manifest folder __/etc/kubernetes/manifests__"
L38: "cluster.go:125: [control-plane] Creating static Pod manifest for __kube-apiserver__"
L39: "cluster.go:125: [control-plane] Creating static Pod manifest for __kube-controller-manager__"
L40: "cluster.go:125: [control-plane] Creating static Pod manifest for __kube-scheduler__"
L41: "cluster.go:125: [kubelet-start] Writing kubelet environment file with flags to file __/var/lib/kubelet/kubeadm-flags.env__"
L42: "cluster.go:125: [kubelet-start] Writing kubelet configuration to file __/var/lib/kubelet/instance-config.yaml__"
L43: "cluster.go:125: [patches] Applied patch of type __application/strategic-merge-patch+json__ to target __kubeletconfiguration__"
L44: "cluster.go:125: [kubelet-start] Writing kubelet configuration to file __/var/lib/kubelet/config.yaml__"
L45: "cluster.go:125: [kubelet-start] Starting the kubelet"
L46: "cluster.go:125: [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory __/etc/kubernetes/manifests__"
L47: "cluster.go:125: [kubelet-check] Waiting for a healthy kubelet at http://127.0.0.1:10248/healthz. This can take up to 4m0s"
L48: "cluster.go:125: [kubelet-check] The kubelet is healthy after 503.481114ms"
L49: "cluster.go:125: [control-plane-check] Waiting for healthy control plane components. This can take up to 30m0s"
L50: "cluster.go:125: [control-plane-check] Checking kube-apiserver at https://10.0.0.10:6443/livez"
L51: "cluster.go:125: [control-plane-check] Checking kube-controller-manager at https://127.0.0.1:10257/healthz"
L52: "cluster.go:125: [control-plane-check] Checking kube-scheduler at https://127.0.0.1:10259/livez"
L53: "cluster.go:125: [control-plane-check] kube-controller-manager is healthy after 1.503329176s"
L54: "cluster.go:125: [control-plane-check] kube-scheduler is healthy after 1.771596015s"
L55: "cluster.go:125: [control-plane-check] kube-apiserver is healthy after 3.501468196s"
L56: "cluster.go:125: [upload-config] Storing the configuration used in ConfigMap __kubeadm-config__ in the __kube-system__ Namespace"
L57: "cluster.go:125: [kubelet] Creating a ConfigMap __kubelet-config__ in namespace kube-system with the configuration for the kubelets in the cluster"
L58: "cluster.go:125: [upload-certs] Skipping phase. Please see --upload-certs"
L59: "cluster.go:125: [mark-control-plane] Marking the node ci-4628.1.2-n-2bebd50bcd as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-lo?ad-balancers]"
L60: "cluster.go:125: [mark-control-plane] Marking the node ci-4628.1.2-n-2bebd50bcd as control-plane by adding the taints [node-role.kubernetes.io/control-plane:NoSchedule]"
L61: "cluster.go:125: [bootstrap-token] Using token: s3asf8.ti8yttpsc3ma17mg"
L62: "cluster.go:125: [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles"
L63: "cluster.go:125: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes"
L64: "cluster.go:125: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials"
L65: "cluster.go:125: [bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token"
L66: "cluster.go:125: [bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster"
L67: "cluster.go:125: [bootstrap-token] Creating the __cluster-info__ ConfigMap in the __kube-public__ namespace"
L68: "cluster.go:125: [kubelet-finalize] Updating __/etc/kubernetes/kubelet.conf__ to point to a rotatable kubelet client certificate and key"
L69: "cluster.go:125: [addons] Applied essential addon: CoreDNS"
L70: "cluster.go:125: [addons] Applied essential addon: kube-proxy"
L71: "cluster.go:125: "
L72: "cluster.go:125: Your Kubernetes control-plane has initialized successfully!"
L73: "cluster.go:125: "
L74: "cluster.go:125: To start using your cluster, you need to run the following as a regular user:"
L75: "cluster.go:125: "
L76: "cluster.go:125: mkdir -p $HOME/.kube"
L77: "cluster.go:125: sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config"
L78: "cluster.go:125: sudo chown $(id -u):$(id -g) $HOME/.kube/config"
L79: "cluster.go:125: "
L80: "cluster.go:125: Alternatively, if you are the root user, you can run:"
L81: "cluster.go:125: "
L82: "cluster.go:125: export KUBECONFIG=/etc/kubernetes/admin.conf"
L83: "cluster.go:125: "
L84: "cluster.go:125: You should now deploy a pod network to the cluster."
L85: "cluster.go:125: Run __kubectl apply -f [podnetwork].yaml__ with one of the options listed at:"
L86: "cluster.go:125: https://kubernetes.io/docs/concepts/cluster-administration/addons/"
L87: "cluster.go:125: "
L88: "cluster.go:125: Then you can join any number of worker nodes by running the following on each as root:"
L89: "cluster.go:125: "
L90: "cluster.go:125: kubeadm join 10.0.0.10:6443 --token s3asf8.ti8yttpsc3ma17mg _"
L91: "cluster.go:125: --discovery-token-ca-cert-hash sha256:a246d496635a1054b089c17f81a7f1883a7e74fc5f934b37e1ef4fa3c706899b "
L92: "cluster.go:125: i Using Cilium version 1.12.5"
L93: "cluster.go:125: ? Auto-detected cluster name: kubernetes"
L94: "cluster.go:125: ? Auto-detected datapath mode: tunnel"
L95: "cluster.go:125: ? Auto-detected kube-proxy has been installed"
L96: "cluster.go:125: i helm template --namespace kube-system cilium cilium/cilium --version 1.12.5 --set cluster.id=0,cluster.name=kubernetes,encryption.nodeEncryption=false,extraConfig.cluster-pool-ipv4-?cidr=192.168.0.0/17,extraConfig.enable-endpoint-routes=true,kubeProxyReplacement=disabled,operator.replicas=1,serviceAccounts.cilium.name=cilium,serviceAccounts.operator.name=cilium-operator,tunnel=vx?lan"
L97: "cluster.go:125: i Storing helm values file in kube-system/cilium-cli-helm-values Secret"
L98: "cluster.go:125: ? Created CA in secret cilium-ca"
L99: "cluster.go:125: ? Generating certificates for Hubble..."
L100: "cluster.go:125: ? Creating Service accounts..."
L101: "cluster.go:125: ? Creating Cluster roles..."
L102: "cluster.go:125: ? Creating ConfigMap for Cilium version 1.12.5..."
L103: "cluster.go:125: i Manual overwrite in ConfigMap: enable-endpoint-routes=true"
L104: "cluster.go:125: i Manual overwrite in ConfigMap: cluster-pool-ipv4-cidr=192.168.0.0/17"
L105: "cluster.go:125: ? Creating Agent DaemonSet..."
L106: "cluster.go:125: level=warning msg=__spec.template.metadata.annotations[container.apparmor.security.beta.kubernetes.io/mount-cgroup]: deprecated since v1.30; use the ___appArmorProfile___ field instead?__ subsys=klog"
L107: "cluster.go:125: level=warning msg=__spec.template.metadata.annotations[container.apparmor.security.beta.kubernetes.io/apply-sysctl-overwrites]: deprecated since v1.30; use the ___appArmorProfile___ fi?eld instead__ subsys=klog"
L108: "cluster.go:125: level=warning msg=__spec.template.metadata.annotations[container.apparmor.security.beta.kubernetes.io/clean-cilium-state]: deprecated since v1.30; use the ___appArmorProfile___ field i?nstead__ subsys=klog"
L109: "cluster.go:125: level=warning msg=__spec.template.metadata.annotations[container.apparmor.security.beta.kubernetes.io/cilium-agent]: deprecated since v1.30; use the ___appArmorProfile___ field instead?__ subsys=klog"
L110: "cluster.go:125: ? Creating Operator Deployment..."
L111: "cluster.go:125: ? Waiting for Cilium to be installed and ready..."
L112: "cluster.go:125: ? Cilium was successfully installed! Run _cilium status_ to view installation health"
L113: "cluster.go:125: ?[33m /??_"
L114: "cluster.go:125: ?[36m /???[33m___/?[32m??_?[0m Cilium: ?[32mOK?[0m"
L115: "cluster.go:125: ?[36m ___?[31m/??_?[32m__/?[0m Operator: ?[32mOK?[0m"
L116: "cluster.go:125: ?[32m /???[31m___/?[35m??_?[0m Hubble: ?[36mdisabled?[0m"
L117: "cluster.go:125: ?[32m ___?[34m/??_?[35m__/?[0m ClusterMesh: ?[36mdisabled?[0m"
L118: "cluster.go:125: ?[34m ___/"
L119: "cluster.go:125: ?[0m"
L120: "cluster.go:125: Deployment cilium-operator "
L121: "cluster.go:125: DaemonSet cilium "
L122: "cluster.go:125: Containers: cilium "
L123: "cluster.go:125: cilium-operator "
L124: "cluster.go:125: Cluster Pods: 0/0 managed by Cilium"
L125: "cluster.go:125: W0604 00:51:25.109964 2597 joinconfiguration.go:112] [config] WARNING: Ignored configuration document with GroupVersionKind kubelet.config.k8s.io/v1beta1, Kind=KubeletConfiguration"
L126: "cluster.go:125: [WARNING Service-kubelet]: kubelet service is not enabled, please run _systemctl enable kubelet.service_"
L127: "cluster.go:125: [preflight] Some fatal errors occurred:"
L128: "cluster.go:125: [ERROR CRI]: could not connect to the container runtime: failed to create new CRI runtime service: validate service connection: validate CRI v1 runtime API for endpoint __unix:///var/?run/containerd/containerd.sock__: rpc error: code = DeadlineExceeded desc = context deadline exceeded while waiting for connections to become ready"
L129: "cluster.go:125: [preflight] If you know what you are doing, you can make a check non-fatal with `--ignore-preflight-errors=...`"
L130: "cluster.go:125: error: error execution phase preflight: preflight checks failed"
L131: "cluster.go:125: To see the stack trace of this error execute with --v=5 or higher"
L132: "kubeadm.go:197: unable to setup cluster: unable to run worker script: Process exited with status 1_"
L133: " "
```
Diagnostic output for azure, run 2
```
L1: " Error: _cluster.go:125: I0604 00:36:39.631829 2811 version.go:260] remote version is much newer: v1.36.1; falling back to: stable-1.35"
L2: "cluster.go:125: [config/images] Pulled registry.k8s.io/kube-apiserver:v1.35.5"
L3: "cluster.go:125: [config/images] Pulled registry.k8s.io/kube-controller-manager:v1.35.5"
L4: "cluster.go:125: [config/images] Pulled registry.k8s.io/kube-scheduler:v1.35.5"
L5: "cluster.go:125: [config/images] Pulled registry.k8s.io/kube-proxy:v1.35.5"
L6: "cluster.go:125: [config/images] Pulled registry.k8s.io/coredns/coredns:v1.13.1"
L7: "cluster.go:125: [config/images] Pulled registry.k8s.io/pause:3.10.1"
L8: "cluster.go:125: [config/images] Pulled registry.k8s.io/etcd:3.6.6-0"
L9: "cluster.go:125: I0604 00:36:50.774874 3138 version.go:260] remote version is much newer: v1.36.1; falling back to: stable-1.35"
L10: "cluster.go:125: [init] Using Kubernetes version: v1.35.5"
L11: "cluster.go:125: [preflight] Running pre-flight checks"
L12: "cluster.go:125: [WARNING Hostname]: hostname __ci-4628.1.2-n-b7d2cb432c__ could not be reached"
L13: "cluster.go:125: [WARNING Hostname]: hostname __ci-4628.1.2-n-b7d2cb432c__: lookup ci-4628.1.2-n-b7d2cb432c on 168.63.129.16:53: no such host"
L14: "cluster.go:125: [WARNING Service-kubelet]: kubelet service is not enabled, please run _systemctl enable kubelet.service_"
L15: "cluster.go:125: [preflight] Pulling images required for setting up a Kubernetes cluster"
L16: "cluster.go:125: [preflight] This might take a minute or two, depending on the speed of your internet connection"
L17: "cluster.go:125: [preflight] You can also perform this action beforehand using _kubeadm config images pull_"
L18: "cluster.go:125: [certs] Using certificateDir folder __/etc/kubernetes/pki__"
L19: "cluster.go:125: [certs] Generating __ca__ certificate and key"
L20: "cluster.go:125: [certs] Generating __apiserver__ certificate and key"
L21: "cluster.go:125: [certs] apiserver serving cert is signed for DNS names [ci-4628.1.2-n-b7d2cb432c kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.?96.0.1 10.0.0.27]"
L22: "cluster.go:125: [certs] Generating __apiserver-kubelet-client__ certificate and key"
L23: "cluster.go:125: [certs] Generating __front-proxy-ca__ certificate and key"
L24: "cluster.go:125: [certs] Generating __front-proxy-client__ certificate and key"
L25: "cluster.go:125: [certs] External etcd mode: Skipping etcd/ca certificate authority generation"
L26: "cluster.go:125: [certs] External etcd mode: Skipping etcd/server certificate generation"
L27: "cluster.go:125: [certs] External etcd mode: Skipping etcd/peer certificate generation"
L28: "cluster.go:125: [certs] External etcd mode: Skipping etcd/healthcheck-client certificate generation"
L29: "cluster.go:125: [certs] External etcd mode: Skipping apiserver-etcd-client certificate generation"
L30: "cluster.go:125: [certs] Generating __sa__ key and public key"
L31: "cluster.go:125: [kubeconfig] Using kubeconfig folder __/etc/kubernetes__"
L32: "cluster.go:125: [kubeconfig] Writing __admin.conf__ kubeconfig file"
L33: "cluster.go:125: [kubeconfig] Writing __super-admin.conf__ kubeconfig file"
L34: "cluster.go:125: [kubeconfig] Writing __kubelet.conf__ kubeconfig file"
L35: "cluster.go:125: [kubeconfig] Writing __controller-manager.conf__ kubeconfig file"
L36: "cluster.go:125: [kubeconfig] Writing __scheduler.conf__ kubeconfig file"
L37: "cluster.go:125: [control-plane] Using manifest folder __/etc/kubernetes/manifests__"
L38: "cluster.go:125: [control-plane] Creating static Pod manifest for __kube-apiserver__"
L39: "cluster.go:125: [control-plane] Creating static Pod manifest for __kube-controller-manager__"
L40: "cluster.go:125: [control-plane] Creating static Pod manifest for __kube-scheduler__"
L41: "cluster.go:125: [kubelet-start] Writing kubelet environment file with flags to file __/var/lib/kubelet/kubeadm-flags.env__"
L42: "cluster.go:125: [kubelet-start] Writing kubelet configuration to file __/var/lib/kubelet/instance-config.yaml__"
L43: "cluster.go:125: [patches] Applied patch of type __application/strategic-merge-patch+json__ to target __kubeletconfiguration__"
L44: "cluster.go:125: [kubelet-start] Writing kubelet configuration to file __/var/lib/kubelet/config.yaml__"
L45: "cluster.go:125: [kubelet-start] Starting the kubelet"
L46: "cluster.go:125: [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory __/etc/kubernetes/manifests__"
L47: "cluster.go:125: [kubelet-check] Waiting for a healthy kubelet at http://127.0.0.1:10248/healthz. This can take up to 4m0s"
L48: "cluster.go:125: [kubelet-check] The kubelet is healthy after 501.831054ms"
L49: "cluster.go:125: [control-plane-check] Waiting for healthy control plane components. This can take up to 30m0s"
L50: "cluster.go:125: [control-plane-check] Checking kube-apiserver at https://10.0.0.27:6443/livez"
L51: "cluster.go:125: [control-plane-check] Checking kube-controller-manager at https://127.0.0.1:10257/healthz"
L52: "cluster.go:125: [control-plane-check] Checking kube-scheduler at https://127.0.0.1:10259/livez"
L53: "cluster.go:125: [control-plane-check] kube-controller-manager is healthy after 1.506450379s"
L54: "cluster.go:125: [control-plane-check] kube-scheduler is healthy after 1.906509279s"
L55: "cluster.go:125: [control-plane-check] kube-apiserver is healthy after 4.000921125s"
L56: "cluster.go:125: [upload-config] Storing the configuration used in ConfigMap __kubeadm-config__ in the __kube-system__ Namespace"
L57: "cluster.go:125: [kubelet] Creating a ConfigMap __kubelet-config__ in namespace kube-system with the configuration for the kubelets in the cluster"
L58: "cluster.go:125: [upload-certs] Skipping phase. Please see --upload-certs"
L59: "cluster.go:125: [mark-control-plane] Marking the node ci-4628.1.2-n-b7d2cb432c as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-lo?ad-balancers]"
L60: "cluster.go:125: [mark-control-plane] Marking the node ci-4628.1.2-n-b7d2cb432c as control-plane by adding the taints [node-role.kubernetes.io/control-plane:NoSchedule]"
L61: "cluster.go:125: [bootstrap-token] Using token: l02p0z.kuanfp4s750d669t"
L62: "cluster.go:125: [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles"
L63: "cluster.go:125: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes"
L64: "cluster.go:125: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials"
L65: "cluster.go:125: [bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token"
L66: "cluster.go:125: [bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster"
L67: "cluster.go:125: [bootstrap-token] Creating the __cluster-info__ ConfigMap in the __kube-public__ namespace"
L68: "cluster.go:125: [kubelet-finalize] Updating __/etc/kubernetes/kubelet.conf__ to point to a rotatable kubelet client certificate and key"
L69: "cluster.go:125: [addons] Applied essential addon: CoreDNS"
L70: "cluster.go:125: [addons] Applied essential addon: kube-proxy"
L71: "cluster.go:125: "
L72: "cluster.go:125: Your Kubernetes control-plane has initialized successfully!"
L73: "cluster.go:125: "
L74: "cluster.go:125: To start using your cluster, you need to run the following as a regular user:"
L75: "cluster.go:125: "
L76: "cluster.go:125: mkdir -p $HOME/.kube"
L77: "cluster.go:125: sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config"
L78: "cluster.go:125: sudo chown $(id -u):$(id -g) $HOME/.kube/config"
L79: "cluster.go:125: "
L80: "cluster.go:125: Alternatively, if you are the root user, you can run:"
L81: "cluster.go:125: "
L82: "cluster.go:125: export KUBECONFIG=/etc/kubernetes/admin.conf"
L83: "cluster.go:125: "
L84: "cluster.go:125: You should now deploy a pod network to the cluster."
L85: "cluster.go:125: Run __kubectl apply -f [podnetwork].yaml__ with one of the options listed at:"
L86: "cluster.go:125: https://kubernetes.io/docs/concepts/cluster-administration/addons/"
L87: "cluster.go:125: "
L88: "cluster.go:125: Then you can join any number of worker nodes by running the following on each as root:"
L89: "cluster.go:125: "
L90: "cluster.go:125: kubeadm join 10.0.0.27:6443 --token l02p0z.kuanfp4s750d669t _"
L91: "cluster.go:125: --discovery-token-ca-cert-hash sha256:bd236696ea24bafd7fa8ff4ded3a402fa1c3e06dbaed04848c0da572d84a5fcc "
L92: "cluster.go:125: i Using Cilium version 1.12.5"
L93: "cluster.go:125: ? Auto-detected cluster name: kubernetes"
L94: "cluster.go:125: ? Auto-detected datapath mode: tunnel"
L95: "cluster.go:125: ? Auto-detected kube-proxy has been installed"
L96: "cluster.go:125: i helm template --namespace kube-system cilium cilium/cilium --version 1.12.5 --set cluster.id=0,cluster.name=kubernetes,encryption.nodeEncryption=false,extraConfig.cluster-pool-ipv4-?cidr=192.168.0.0/17,extraConfig.enable-endpoint-routes=true,kubeProxyReplacement=disabled,operator.replicas=1,serviceAccounts.cilium.name=cilium,serviceAccounts.operator.name=cilium-operator,tunnel=vx?lan"
L97: "cluster.go:125: i Storing helm values file in kube-system/cilium-cli-helm-values Secret"
L98: "cluster.go:125: ? Created CA in secret cilium-ca"
L99: "cluster.go:125: ? Generating certificates for Hubble..."
L100: "cluster.go:125: ? Creating Service accounts..."
L101: "cluster.go:125: ? Creating Cluster roles..."
L102: "cluster.go:125: ? Creating ConfigMap for Cilium version 1.12.5..."
L103: "cluster.go:125: i Manual overwrite in ConfigMap: enable-endpoint-routes=true"
L104: "cluster.go:125: i Manual overwrite in ConfigMap: cluster-pool-ipv4-cidr=192.168.0.0/17"
L105: "cluster.go:125: ? Creating Agent DaemonSet..."
L106: "cluster.go:125: level=warning msg=__spec.template.metadata.annotations[container.apparmor.security.beta.kubernetes.io/mount-cgroup]: deprecated since v1.30; use the ___appArmorProfile___ field instead?__ subsys=klog"
L107: "cluster.go:125: level=warning msg=__spec.template.metadata.annotations[container.apparmor.security.beta.kubernetes.io/apply-sysctl-overwrites]: deprecated since v1.30; use the ___appArmorProfile___ fi?eld instead__ subsys=klog"
L108: "cluster.go:125: level=warning msg=__spec.template.metadata.annotations[container.apparmor.security.beta.kubernetes.io/clean-cilium-state]: deprecated since v1.30; use the ___appArmorProfile___ field i?nstead__ subsys=klog"
L109: "cluster.go:125: level=warning msg=__spec.template.metadata.annotations[container.apparmor.security.beta.kubernetes.io/cilium-agent]: deprecated since v1.30; use the ___appArmorProfile___ field instead?__ subsys=klog"
L110: "cluster.go:125: ? Creating Operator Deployment..."
L111: "cluster.go:125: ? Waiting for Cilium to be installed and ready..."
L112: "cluster.go:125: ? Cilium was successfully installed! Run _cilium status_ to view installation health"
L113: "cluster.go:125: ?[33m /??_"
L114: "cluster.go:125: ?[36m /???[33m___/?[32m??_?[0m Cilium: ?[32mOK?[0m"
L115: "cluster.go:125: ?[36m ___?[31m/??_?[32m__/?[0m Operator: ?[32mOK?[0m"
L116: "cluster.go:125: ?[32m /???[31m___/?[35m??_?[0m Hubble: ?[36mdisabled?[0m"
L117: "cluster.go:125: ?[32m ___?[34m/??_?[35m__/?[0m ClusterMesh: ?[36mdisabled?[0m"
L118: "cluster.go:125: ?[34m ___/"
L119: "cluster.go:125: ?[0m"
L120: "cluster.go:125: Deployment cilium-operator "
L121: "cluster.go:125: DaemonSet cilium "
L122: "cluster.go:125: Containers: cilium "
L123: "cluster.go:125: cilium-operator "
L124: "cluster.go:125: Cluster Pods: 0/0 managed by Cilium"
L125: "kubeadm.go:197: unable to setup cluster: unable to create worker node: PollUntilDone(ci-4628.1.2-n-7b8d6360a8): GET https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/prov?iders/Microsoft.Compute/locations/westus2/operations/134aabd0-f9be-4006-8962-833c482089eb"
L126: "--------------------------------------------------------------------------------"
L127: "RESPONSE 200: 200 OK"
L128: "ERROR CODE: OperationPreempted"
L129: "--------------------------------------------------------------------------------"
L130: "{"
L131: "__startTime__: __2026-06-04T00:37:06.9899748+00:00__,"
L132: "__endTime__: __2026-06-04T00:38:41.5771522+00:00__,"
L133: "__status__: __Canceled__,"
L134: "__error__: {"
L135: "__code__: __OperationPreempted__,"
L136: "__message__: __Operation execution has been preempted by a more recent operation.__"
L137: "},"
L138: "__name__: __134aabd0-f9be-4006-8962-833c482089eb__"
L139: "}"
L140: "--------------------------------------------------------------------------------_"
L141: " "
```
Diagnostic output for azure, run 1
```
L1: " Error: _kubeadm.go:197: unable to setup cluster: unable to create etcd node: creating public ip: PUT https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kola?-cluster-image-67e972cd9c/providers/Microsoft.Network/publicIPAddresses/ip-8684fe545f"
L2: "--------------------------------------------------------------------------------"
L3: "RESPONSE 409: 409 Conflict"
L4: "ERROR CODE: ResourceGroupBeingDeleted"
L5: "--------------------------------------------------------------------------------"
L6: "{"
L7: "__error__: {"
L8: "__code__: __ResourceGroupBeingDeleted__,"
L9: "__message__: __The resource group _kola-cluster-image-67e972cd9c_ is in deprovisioning state and cannot perform this operation.__"
L10: "}"
L11: "}"
L12: "--------------------------------------------------------------------------------_"
L13: " "
```
🟢 ok **kubeadm.v1.35.1.flannel.base**; Succeeded: azure (2); Failed: azure (1)
Diagnostic output for azure, run 1
```
L1: " Error: _kubeadm.go:197: unable to setup cluster: unable to create master node: creating public ip: PUT https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/ko?la-cluster-image-67e972cd9c/providers/Microsoft.Network/publicIPAddresses/ip-241c4e486b"
L2: "--------------------------------------------------------------------------------"
L3: "RESPONSE 409: 409 Conflict"
L4: "ERROR CODE: ResourceGroupBeingDeleted"
L5: "--------------------------------------------------------------------------------"
L6: "{"
L7: "__error__: {"
L8: "__code__: __ResourceGroupBeingDeleted__,"
L9: "__message__: __The resource group _kola-cluster-image-67e972cd9c_ is in deprovisioning state and cannot perform this operation.__"
L10: "}"
L11: "}"
L12: "--------------------------------------------------------------------------------_"
L13: " "
```
🟢 ok **kubeadm.v1.36.1.calico.base**; Succeeded: azure (3); Failed: azure (1, 2)
Diagnostic output for azure, run 2
```
L1: " Error: _cluster.go:125: [config/images] Pulled registry.k8s.io/kube-apiserver:v1.36.1"
L2: "cluster.go:125: [config/images] Pulled registry.k8s.io/kube-controller-manager:v1.36.1"
L3: "cluster.go:125: [config/images] Pulled registry.k8s.io/kube-scheduler:v1.36.1"
L4: "cluster.go:125: [config/images] Pulled registry.k8s.io/kube-proxy:v1.36.1"
L5: "cluster.go:125: [config/images] Pulled registry.k8s.io/coredns/coredns:v1.14.2"
L6: "cluster.go:125: [config/images] Pulled registry.k8s.io/pause:3.10.2"
L7: "cluster.go:125: [config/images] Pulled registry.k8s.io/etcd:3.6.8-0"
L8: "cluster.go:125: [init] Using Kubernetes version: v1.36.1"
L9: "cluster.go:125: [preflight] Running pre-flight checks"
L10: "cluster.go:125: [WARNING Hostname]: hostname __ci-4628.1.2-n-9bb18ebb0d__ could not be reached"
L11: "cluster.go:125: [WARNING Hostname]: hostname __ci-4628.1.2-n-9bb18ebb0d__: lookup ci-4628.1.2-n-9bb18ebb0d on 168.63.129.16:53: no such host"
L12: "cluster.go:125: [WARNING Service-kubelet]: kubelet service is not enabled, please run _systemctl enable kubelet.service_"
L13: "cluster.go:125: [preflight] Pulling images required for setting up a Kubernetes cluster"
L14: "cluster.go:125: [preflight] This might take a minute or two, depending on the speed of your internet connection"
L15: "cluster.go:125: [preflight] You can also perform this action beforehand using _kubeadm config images pull_"
L16: "cluster.go:125: [certs] Using certificateDir folder __/etc/kubernetes/pki__"
L17: "cluster.go:125: [certs] Generating __ca__ certificate and key"
L18: "cluster.go:125: [certs] Generating __apiserver__ certificate and key"
L19: "cluster.go:125: [certs] apiserver serving cert is signed for DNS names [ci-4628.1.2-n-9bb18ebb0d kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.?96.0.1 10.0.0.26]"
L20: "cluster.go:125: [certs] Generating __apiserver-kubelet-client__ certificate and key"
L21: "cluster.go:125: [certs] Generating __front-proxy-ca__ certificate and key"
L22: "cluster.go:125: [certs] Generating __front-proxy-client__ certificate and key"
L23: "cluster.go:125: [certs] External etcd mode: Skipping etcd/ca certificate authority generation"
L24: "cluster.go:125: [certs] External etcd mode: Skipping etcd/server certificate generation"
L25: "cluster.go:125: [certs] External etcd mode: Skipping etcd/peer certificate generation"
L26: "cluster.go:125: [certs] External etcd mode: Skipping etcd/healthcheck-client certificate generation"
L27: "cluster.go:125: [certs] External etcd mode: Skipping apiserver-etcd-client certificate generation"
L28: "cluster.go:125: [certs] Generating __sa__ key and public key"
L29: "cluster.go:125: [kubeconfig] Using kubeconfig folder __/etc/kubernetes__"
L30: "cluster.go:125: [kubeconfig] Writing __admin.conf__ kubeconfig file"
L31: "cluster.go:125: [kubeconfig] Writing __super-admin.conf__ kubeconfig file"
L32: "cluster.go:125: [kubeconfig] Writing __kubelet.conf__ kubeconfig file"
L33: "cluster.go:125: [kubeconfig] Writing __controller-manager.conf__ kubeconfig file"
L34: "cluster.go:125: [kubeconfig] Writing __scheduler.conf__ kubeconfig file"
L35: "cluster.go:125: [control-plane] Using manifest folder __/etc/kubernetes/manifests__"
L36: "cluster.go:125: [control-plane] Creating static Pod manifest for __kube-apiserver__"
L37: "cluster.go:125: [control-plane] Creating static Pod manifest for __kube-controller-manager__"
L38: "cluster.go:125: [control-plane] Creating static Pod manifest for __kube-scheduler__"
L39: "cluster.go:125: [kubelet-start] Writing kubelet environment file with flags to file __/var/lib/kubelet/kubeadm-flags.env__"
L40: "cluster.go:125: [kubelet-start] Writing kubelet configuration to file __/var/lib/kubelet/instance-config.yaml__"
L41: "cluster.go:125: [patches] Applied patch of type __application/strategic-merge-patch+json__ to target __kubeletconfiguration__"
L42: "cluster.go:125: [kubelet-start] Writing kubelet configuration to file __/var/lib/kubelet/config.yaml__"
L43: "cluster.go:125: [kubelet-start] Starting the kubelet"
L44: "cluster.go:125: [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory __/etc/kubernetes/manifests__"
L45: "cluster.go:125: [kubelet-check] Waiting for a healthy kubelet at http://127.0.0.1:10248/healthz. This can take up to 4m0s"
L46: "cluster.go:125: [kubelet-check] The kubelet is healthy after 646.798??s"
L47: "cluster.go:125: [control-plane-check] Waiting for healthy control plane components. This can take up to 30m0s"
L48: "cluster.go:125: [control-plane-check] Checking kube-apiserver at https://10.0.0.26:6443/livez"
L49: "cluster.go:125: [control-plane-check] Checking kube-controller-manager at https://127.0.0.1:10257/healthz"
L50: "cluster.go:125: [control-plane-check] Checking kube-scheduler at https://127.0.0.1:10259/livez"
L51: "cluster.go:125: [control-plane-check] kube-scheduler is healthy after 4.134857ms"
L52: "cluster.go:125: [control-plane-check] kube-controller-manager is healthy after 6.104475ms"
L53: "cluster.go:125: [control-plane-check] kube-apiserver is healthy after 2.002832347s"
L54: "cluster.go:125: [upload-config] Storing the configuration used in ConfigMap __kubeadm-config__ in the __kube-system__ Namespace"
L55: "cluster.go:125: [kubelet] Creating a ConfigMap __kubelet-config__ in namespace kube-system with the configuration for the kubelets in the cluster"
L56: "cluster.go:125: [upload-certs] Skipping phase. Please see --upload-certs"
L57: "cluster.go:125: [mark-control-plane] Marking the node ci-4628.1.2-n-9bb18ebb0d as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-lo?ad-balancers]"
L58: "cluster.go:125: [mark-control-plane] Marking the node ci-4628.1.2-n-9bb18ebb0d as control-plane by adding the taints [node-role.kubernetes.io/control-plane:NoSchedule]"
L59: "cluster.go:125: [bootstrap-token] Using token: vh3zbl.77zm4toe0jy6069g"
L60: "cluster.go:125: [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles"
L61: "cluster.go:125: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes"
L62: "cluster.go:125: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials"
L63: "cluster.go:125: [bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token"
L64: "cluster.go:125: [bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster"
L65: "cluster.go:125: [bootstrap-token] Configured RBAC rules to allow the API server kubelet client certificate to access the kubelet API"
L66: "cluster.go:125: [bootstrap-token] Creating the __cluster-info__ ConfigMap in the __kube-public__ namespace"
L67: "cluster.go:125: [kubelet-finalize] Updating __/etc/kubernetes/kubelet.conf__ to point to a rotatable kubelet client certificate and key"
L68: "cluster.go:125: [addons] Applied essential addon: CoreDNS"
L69: "cluster.go:125: [addons] Applied essential addon: kube-proxy"
L70: "cluster.go:125: "
L71: "cluster.go:125: Your Kubernetes control-plane has initialized successfully!"
L72: "cluster.go:125: "
L73: "cluster.go:125: To start using your cluster, you need to run the following as a regular user:"
L74: "cluster.go:125: "
L75: "cluster.go:125: mkdir -p $HOME/.kube"
L76: "cluster.go:125: sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config"
L77: "cluster.go:125: sudo chown $(id -u):$(id -g) $HOME/.kube/config"
L78: "cluster.go:125: "
L79: "cluster.go:125: Alternatively, if you are the root user, you can run:"
L80: "cluster.go:125: "
L81: "cluster.go:125: export KUBECONFIG=/etc/kubernetes/admin.conf"
L82: "cluster.go:125: "
L83: "cluster.go:125: You should now deploy a pod network to the cluster."
L84: "cluster.go:125: Run __kubectl apply -f [podnetwork].yaml__ with one of the options listed at:"
L85: "cluster.go:125: https://kubernetes.io/docs/concepts/cluster-administration/addons/"
L86: "cluster.go:125: "
L87: "cluster.go:125: Then you can join any number of worker nodes by running the following on each as root:"
L88: "cluster.go:125: "
L89: "cluster.go:125: kubeadm join 10.0.0.26:6443 --token vh3zbl.77zm4toe0jy6069g _"
L90: "cluster.go:125: --discovery-token-ca-cert-hash sha256:869a14d8508683189aa4e10a3acf0ec9f04d035e1fbf997035f0f50bb58137dc "
L91: "cluster.go:125: namespace/tigera-operator created"
L92: "cluster.go:125: serviceaccount/tigera-operator created"
L93: "cluster.go:125: clusterrole.rbac.authorization.k8s.io/tigera-operator-secrets created"
L94: "cluster.go:125: clusterrole.rbac.authorization.k8s.io/tigera-operator created"
L95: "cluster.go:125: clusterrolebinding.rbac.authorization.k8s.io/tigera-operator created"
L96: "cluster.go:125: rolebinding.rbac.authorization.k8s.io/tigera-operator-secrets created"
L97: "cluster.go:125: deployment.apps/tigera-operator created"
L98: "cluster.go:125: customresourcedefinition.apiextensions.k8s.io/installations.operator.tigera.io condition met"
L99: "cluster.go:125: customresourcedefinition.apiextensions.k8s.io/installations.operator.tigera.io condition met"
L100: "cluster.go:125: customresourcedefinition.apiextensions.k8s.io/apiservers.operator.tigera.io condition met"
L101: "cluster.go:125: customresourcedefinition.apiextensions.k8s.io/apiservers.operator.tigera.io condition met"
L102: "cluster.go:125: installation.operator.tigera.io/default created"
L103: "cluster.go:125: apiserver.operator.tigera.io/default created"
L104: "cluster.go:125: goldmane.operator.tigera.io/default created"
L105: "cluster.go:125: whisker.operator.tigera.io/default created"
L106: "kubeadm.go:197: unable to setup cluster: unable to create worker node: PollUntilDone(ci-4628.1.2-n-843103bd35): GET https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/prov?iders/Microsoft.Compute/locations/westus2/operations/74c44fc2-5502-4e30-b61a-4c50bfa9c223"
L107: "--------------------------------------------------------------------------------"
L108: "RESPONSE 200: 200 OK"
L109: "ERROR CODE: OperationPreempted"
L110: "--------------------------------------------------------------------------------"
L111: "{"
L112: "__startTime__: __2026-06-04T00:37:13.9220817+00:00__,"
L113: "__endTime__: __2026-06-04T00:38:41.6833102+00:00__,"
L114: "__status__: __Canceled__,"
L115: "__error__: {"
L116: "__code__: __OperationPreempted__,"
L117: "__message__: __Operation execution has been preempted by a more recent operation.__"
L118: "},"
L119: "__name__: __74c44fc2-5502-4e30-b61a-4c50bfa9c223__"
L120: "}"
L121: "--------------------------------------------------------------------------------_"
L122: " "
```
Diagnostic output for azure, run 1
```
L1: " Error: _kubeadm.go:197: unable to setup cluster: unable to create master node: creating public ip: PUT https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/ko?la-cluster-image-67e972cd9c/providers/Microsoft.Network/publicIPAddresses/ip-24d2475206"
L2: "--------------------------------------------------------------------------------"
L3: "RESPONSE 409: 409 Conflict"
L4: "ERROR CODE: ResourceGroupBeingDeleted"
L5: "--------------------------------------------------------------------------------"
L6: "{"
L7: "__error__: {"
L8: "__code__: __ResourceGroupBeingDeleted__,"
L9: "__message__: __The resource group _kola-cluster-image-67e972cd9c_ is in deprovisioning state and cannot perform this operation.__"
L10: "}"
L11: "}"
L12: "--------------------------------------------------------------------------------_"
L13: " "
```
🟢 ok **kubeadm.v1.36.1.cilium.base**; Succeeded: azure (2); Failed: azure (1)
Diagnostic output for azure, run 1
```
L1: " Error: _kubeadm.go:197: unable to setup cluster: unable to create etcd node: creating public ip: PUT https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kola?-cluster-image-67e972cd9c/providers/Microsoft.Network/publicIPAddresses/ip-6b4b245231"
L2: "--------------------------------------------------------------------------------"
L3: "RESPONSE 409: 409 Conflict"
L4: "ERROR CODE: ResourceGroupBeingDeleted"
L5: "--------------------------------------------------------------------------------"
L6: "{"
L7: "__error__: {"
L8: "__code__: __ResourceGroupBeingDeleted__,"
L9: "__message__: __The resource group _kola-cluster-image-67e972cd9c_ is in deprovisioning state and cannot perform this operation.__"
L10: "}"
L11: "}"
L12: "--------------------------------------------------------------------------------_"
L13: " "
```
🟢 ok **kubeadm.v1.36.1.flannel.base**; Succeeded: azure (3); Failed: azure (1, 2)
Diagnostic output for azure, run 2
```
L1: " Error: _cluster.go:125: [config/images] Pulled registry.k8s.io/kube-apiserver:v1.36.1"
L2: "cluster.go:125: [config/images] Pulled registry.k8s.io/kube-controller-manager:v1.36.1"
L3: "cluster.go:125: [config/images] Pulled registry.k8s.io/kube-scheduler:v1.36.1"
L4: "cluster.go:125: [config/images] Pulled registry.k8s.io/kube-proxy:v1.36.1"
L5: "cluster.go:125: [config/images] Pulled registry.k8s.io/coredns/coredns:v1.14.2"
L6: "cluster.go:125: [config/images] Pulled registry.k8s.io/pause:3.10.2"
L7: "cluster.go:125: [config/images] Pulled registry.k8s.io/etcd:3.6.8-0"
L8: "cluster.go:125: [init] Using Kubernetes version: v1.36.1"
L9: "cluster.go:125: [preflight] Running pre-flight checks"
L10: "cluster.go:125: [WARNING Hostname]: hostname __ci-4628.1.2-n-8636e79a14__ could not be reached"
L11: "cluster.go:125: [WARNING Hostname]: hostname __ci-4628.1.2-n-8636e79a14__: lookup ci-4628.1.2-n-8636e79a14 on 168.63.129.16:53: no such host"
L12: "cluster.go:125: [WARNING Service-kubelet]: kubelet service is not enabled, please run _systemctl enable kubelet.service_"
L13: "cluster.go:125: [preflight] Pulling images required for setting up a Kubernetes cluster"
L14: "cluster.go:125: [preflight] This might take a minute or two, depending on the speed of your internet connection"
L15: "cluster.go:125: [preflight] You can also perform this action beforehand using _kubeadm config images pull_"
L16: "cluster.go:125: [certs] Using certificateDir folder __/etc/kubernetes/pki__"
L17: "cluster.go:125: [certs] Generating __ca__ certificate and key"
L18: "cluster.go:125: [certs] Generating __apiserver__ certificate and key"
L19: "cluster.go:125: [certs] apiserver serving cert is signed for DNS names [ci-4628.1.2-n-8636e79a14 kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.?96.0.1 10.0.0.11]"
L20: "cluster.go:125: [certs] Generating __apiserver-kubelet-client__ certificate and key"
L21: "cluster.go:125: [certs] Generating __front-proxy-ca__ certificate and key"
L22: "cluster.go:125: [certs] Generating __front-proxy-client__ certificate and key"
L23: "cluster.go:125: [certs] External etcd mode: Skipping etcd/ca certificate authority generation"
L24: "cluster.go:125: [certs] External etcd mode: Skipping etcd/server certificate generation"
L25: "cluster.go:125: [certs] External etcd mode: Skipping etcd/peer certificate generation"
L26: "cluster.go:125: [certs] External etcd mode: Skipping etcd/healthcheck-client certificate generation"
L27: "cluster.go:125: [certs] External etcd mode: Skipping apiserver-etcd-client certificate generation"
L28: "cluster.go:125: [certs] Generating __sa__ key and public key"
L29: "cluster.go:125: [kubeconfig] Using kubeconfig folder __/etc/kubernetes__"
L30: "cluster.go:125: [kubeconfig] Writing __admin.conf__ kubeconfig file"
L31: "cluster.go:125: [kubeconfig] Writing __super-admin.conf__ kubeconfig file"
L32: "cluster.go:125: [kubeconfig] Writing __kubelet.conf__ kubeconfig file"
L33: "cluster.go:125: [kubeconfig] Writing __controller-manager.conf__ kubeconfig file"
L34: "cluster.go:125: [kubeconfig] Writing __scheduler.conf__ kubeconfig file"
L35: "cluster.go:125: [control-plane] Using manifest folder __/etc/kubernetes/manifests__"
L36: "cluster.go:125: [control-plane] Creating static Pod manifest for __kube-apiserver__"
L37: "cluster.go:125: [control-plane] Creating static Pod manifest for __kube-controller-manager__"
L38: "cluster.go:125: [control-plane] Creating static Pod manifest for __kube-scheduler__"
L39: "cluster.go:125: [kubelet-start] Writing kubelet environment file with flags to file __/var/lib/kubelet/kubeadm-flags.env__"
L40: "cluster.go:125: [kubelet-start] Writing kubelet configuration to file __/var/lib/kubelet/instance-config.yaml__"
L41: "cluster.go:125: [patches] Applied patch of type __application/strategic-merge-patch+json__ to target __kubeletconfiguration__"
L42: "cluster.go:125: [kubelet-start] Writing kubelet configuration to file __/var/lib/kubelet/config.yaml__"
L43: "cluster.go:125: [kubelet-start] Starting the kubelet"
L44: "cluster.go:125: [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory __/etc/kubernetes/manifests__"
L45: "cluster.go:125: [kubelet-check] Waiting for a healthy kubelet at http://127.0.0.1:10248/healthz. This can take up to 4m0s"
L46: "cluster.go:125: [kubelet-check] The kubelet is healthy after 616.595??s"
L47: "cluster.go:125: [control-plane-check] Waiting for healthy control plane components. This can take up to 30m0s"
L48: "cluster.go:125: [control-plane-check] Checking kube-apiserver at https://10.0.0.11:6443/livez"
L49: "cluster.go:125: [control-plane-check] Checking kube-controller-manager at https://127.0.0.1:10257/healthz"
L50: "cluster.go:125: [control-plane-check] Checking kube-scheduler at https://127.0.0.1:10259/livez"
L51: "cluster.go:125: [control-plane-check] kube-controller-manager is healthy after 5.261044ms"
L52: "cluster.go:125: [control-plane-check] kube-scheduler is healthy after 8.198818ms"
L53: "cluster.go:125: [control-plane-check] kube-apiserver is healthy after 1.501694139s"
L54: "cluster.go:125: [upload-config] Storing the configuration used in ConfigMap __kubeadm-config__ in the __kube-system__ Namespace"
L55: "cluster.go:125: [kubelet] Creating a ConfigMap __kubelet-config__ in namespace kube-system with the configuration for the kubelets in the cluster"
L56: "cluster.go:125: [upload-certs] Skipping phase. Please see --upload-certs"
L57: "cluster.go:125: [mark-control-plane] Marking the node ci-4628.1.2-n-8636e79a14 as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-lo?ad-balancers]"
L58: "cluster.go:125: [mark-control-plane] Marking the node ci-4628.1.2-n-8636e79a14 as control-plane by adding the taints [node-role.kubernetes.io/control-plane:NoSchedule]"
L59: "cluster.go:125: [bootstrap-token] Using token: yyz9qu.jsc8x8ee8fn2sg7x"
L60: "cluster.go:125: [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles"
L61: "cluster.go:125: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes"
L62: "cluster.go:125: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials"
L63: "cluster.go:125: [bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token"
L64: "cluster.go:125: [bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster"
L65: "cluster.go:125: [bootstrap-token] Configured RBAC rules to allow the API server kubelet client certificate to access the kubelet API"
L66: "cluster.go:125: [bootstrap-token] Creating the __cluster-info__ ConfigMap in the __kube-public__ namespace"
L67: "cluster.go:125: [kubelet-finalize] Updating __/etc/kubernetes/kubelet.conf__ to point to a rotatable kubelet client certificate and key"
L68: "cluster.go:125: [addons] Applied essential addon: CoreDNS"
L69: "cluster.go:125: [addons] Applied essential addon: kube-proxy"
L70: "cluster.go:125: "
L71: "cluster.go:125: Your Kubernetes control-plane has initialized successfully!"
L72: "cluster.go:125: "
L73: "cluster.go:125: To start using your cluster, you need to run the following as a regular user:"
L74: "cluster.go:125: "
L75: "cluster.go:125: mkdir -p $HOME/.kube"
L76: "cluster.go:125: sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config"
L77: "cluster.go:125: sudo chown $(id -u):$(id -g) $HOME/.kube/config"
L78: "cluster.go:125: "
L79: "cluster.go:125: Alternatively, if you are the root user, you can run:"
L80: "cluster.go:125: "
L81: "cluster.go:125: export KUBECONFIG=/etc/kubernetes/admin.conf"
L82: "cluster.go:125: "
L83: "cluster.go:125: You should now deploy a pod network to the cluster."
L84: "cluster.go:125: Run __kubectl apply -f [podnetwork].yaml__ with one of the options listed at:"
L85: "cluster.go:125: https://kubernetes.io/docs/concepts/cluster-administration/addons/"
L86: "cluster.go:125: "
L87: "cluster.go:125: Then you can join any number of worker nodes by running the following on each as root:"
L88: "cluster.go:125: "
L89: "cluster.go:125: kubeadm join 10.0.0.11:6443 --token yyz9qu.jsc8x8ee8fn2sg7x _"
L90: "cluster.go:125: --discovery-token-ca-cert-hash sha256:a1237fb8e0050a25d9556665610d172497cbd64ea73a23d50d8eff8781326c5d "
L91: "cluster.go:125: namespace/kube-flannel created"
L92: "cluster.go:125: clusterrole.rbac.authorization.k8s.io/flannel created"
L93: "cluster.go:125: clusterrolebinding.rbac.authorization.k8s.io/flannel created"
L94: "cluster.go:125: serviceaccount/flannel created"
L95: "cluster.go:125: configmap/kube-flannel-cfg created"
L96: "cluster.go:125: daemonset.apps/kube-flannel-ds created"
L97: "kubeadm.go:197: unable to setup cluster: unable to create worker node: PollUntilDone(ci-4628.1.2-n-13a0542c81): GET https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/prov?iders/Microsoft.Compute/locations/westus2/operations/b49cf512-5de3-4135-ba41-be35f7f1fea9"
L98: "--------------------------------------------------------------------------------"
L99: "RESPONSE 200: 200 OK"
L100: "ERROR CODE: OperationPreempted"
L101: "--------------------------------------------------------------------------------"
L102: "{"
L103: "__startTime__: __2026-06-04T00:37:16.7991389+00:00__,"
L104: "__endTime__: __2026-06-04T00:38:34.9123015+00:00__,"
L105: "__status__: __Canceled__,"
L106: "__error__: {"
L107: "__code__: __OperationPreempted__,"
L108: "__message__: __Operation execution has been preempted by a more recent operation.__"
L109: "},"
L110: "__name__: __b49cf512-5de3-4135-ba41-be35f7f1fea9__"
L111: "}"
L112: "--------------------------------------------------------------------------------_"
L113: " "
```
Diagnostic output for azure, run 1
```
L1: " Error: _kubeadm.go:197: unable to setup cluster: unable to create master node: creating public ip: PUT https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/ko?la-cluster-image-67e972cd9c/providers/Microsoft.Network/publicIPAddresses/ip-cc3374ed3f"
L2: "--------------------------------------------------------------------------------"
L3: "RESPONSE 409: 409 Conflict"
L4: "ERROR CODE: ResourceGroupBeingDeleted"
L5: "--------------------------------------------------------------------------------"
L6: "{"
L7: "__error__: {"
L8: "__code__: __ResourceGroupBeingDeleted__,"
L9: "__message__: __The resource group _kola-cluster-image-67e972cd9c_ is in deprovisioning state and cannot perform this operation.__"
L10: "}"
L11: "}"
L12: "--------------------------------------------------------------------------------_"
L13: " "
```
🟢 ok **sysext.custom-docker.sysext**; Succeeded: azure (2); Failed: azure (1)
Diagnostic output for azure, run 1
```
L1: " Error: _harness.go:646: Cluster failed starting machines: creating public ip: PUT https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kola-cluster-image-67e9?72cd9c/providers/Microsoft.Network/publicIPAddresses/ip-03d8b4b3c1"
L2: "--------------------------------------------------------------------------------"
L3: "RESPONSE 409: 409 Conflict"
L4: "ERROR CODE: ResourceGroupBeingDeleted"
L5: "--------------------------------------------------------------------------------"
L6: "{"
L7: "__error__: {"
L8: "__code__: __ResourceGroupBeingDeleted__,"
L9: "__message__: __The resource group _kola-cluster-image-67e972cd9c_ is in deprovisioning state and cannot perform this operation.__"
L10: "}"
L11: "}"
L12: "--------------------------------------------------------------------------------_"
L13: " "
```
🟢 ok **sysext.disable-containerd**; Succeeded: azure (1)
🟢 ok **sysext.disable-docker**; Succeeded: azure (1)
🟢 ok **sysext.simple**; Succeeded: azure (2); Failed: azure (1)
Diagnostic output for azure, run 1
```
L1: " Error: _harness.go:646: Cluster failed starting machines: waiting for machine to become active: GET https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kola-?cluster-image-67e972cd9c/providers/Microsoft.Compute/virtualMachines/ci-4628.1.2-n-0c0ff794f8"
L2: "--------------------------------------------------------------------------------"
L3: "RESPONSE 404: 404 Not Found"
L4: "ERROR CODE: NotFound"
L5: "--------------------------------------------------------------------------------"
L6: "{"
L7: "__error__: {"
L8: "__code__: __NotFound__,"
L9: "__message__: __The entity was not found in this Azure location.__"
L10: "}"
L11: "}"
L12: "--------------------------------------------------------------------------------_"
L13: " "
```
🟢 ok **systemd.journal.user**; Succeeded: azure (1)
🟢 ok **systemd.sysusers.gshadow**; Succeeded: azure (1)