### Test report for 4081.3.9+nightly-20260724-2100 / amd64 **Platforms tested** : azure 🟢 ok **bpf.ig**; Succeeded: azure (1) 🟢 ok **cl.basic**; Succeeded: azure (1) 🟢 ok **cl.cloudinit.basic**; Succeeded: azure (1) 🟢 ok **cl.cloudinit.multipart-mime**; Succeeded: azure (1) 🟢 ok **cl.cloudinit.script**; Succeeded: azure (1) 🟢 ok **cl.etcd-member.discovery**; Succeeded: azure (1) 🟢 ok **cl.etcd-member.etcdctlv3**; Succeeded: azure (1) 🟢 ok **cl.etcd-member.v2-backup-restore**; Succeeded: azure (1) 🟢 ok **cl.flannel.udp**; Succeeded: azure (2); Failed: azure (1)
                Diagnostic output for azure, run 1 ``` L1: " Error: _harness.go:646: Cluster failed starting machines: PollUntilDone(ci-4081.3.9-n-83189f5081): GET https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/providers/Microso?ft.Compute/locations/northeurope/operations/16721953-4718-4c23-94aa-cb511e96c7fc" L2: "--------------------------------------------------------------------------------" L3: "RESPONSE 200: 200 OK" L4: "ERROR CODE: OperationPreempted" L5: "--------------------------------------------------------------------------------" L6: "{" L7: "__startTime__: __2026-07-24T23:43:48.6889805+00:00__," L8: "__endTime__: __2026-07-24T23:44:56.3626535+00:00__," L9: "__status__: __Canceled__," L10: "__error__: {" L11: "__code__: __OperationPreempted__," L12: "__message__: __Operation execution has been preempted by a more recent operation.__" L13: "}," L14: "__name__: __16721953-4718-4c23-94aa-cb511e96c7fc__" L15: "}" L16: "--------------------------------------------------------------------------------_" L17: " " ```
🟢 ok **cl.flannel.vxlan**; Succeeded: azure (1) 🟢 ok **cl.ignition.kargs**; Succeeded: azure (2); Failed: azure (1)
                Diagnostic output for azure, run 1 ``` L1: " Error: _harness.go:646: Cluster failed starting machines: PUT https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kola-cluster-image-b4ba8baeca/providers/Mic?rosoft.Compute/virtualMachines/ci-4081.3.9-n-ebd2a6141f" L2: "--------------------------------------------------------------------------------" L3: "RESPONSE 409: 409 Conflict" L4: "ERROR CODE: OperationNotAllowed" L5: "--------------------------------------------------------------------------------" L6: "{" L7: "__error__: {" L8: "__code__: __OperationNotAllowed__," L9: "__message__: __Operation could not be completed as it results in exceeding approved standardDSv4Family Cores quota. Additional details - Deployment Model: Resource Manager, Location: northeurope, Curr?ent Limit: 65, Current Usage: 64, Additional Required: 2, (Minimum) New Limit Required: 66. Setup Alerts when Quota reaches threshold. Learn more at https://aka.ms/quotamonitoringalerting . Submit a r?equest for Quota increase at https://aka.ms/ProdportalCRP/#blade/Microsoft_Azure_Capacity/UsageAndQuota.ReactView/Parameters/%7B%22subscriptionId%22:%220e46bd28-a80f-4d3a-8200-d9eb8d80cb2e%22,%22comma?nd%22:%22openQuotaApprovalBlade%22,%22quotas%22:[%7B%22location%22:%22northeurope%22,%22providerId%22:%22Microsoft.Compute%22,%22resourceName%22:%22standardDSv4Family%22,%22quotaRequest%22:%7B%22prope?rties%22:%7B%22limit%22:66,%22unit%22:%22Count%22,%22name%22:%7B%22value%22:%22standardDSv4Family%22%7D%7D%7D%7D]%7D by specifying parameters listed in the ???Details??? section for deployment to succ?eed. Please read more about quota limits at https://docs.microsoft.com/en-us/azure/azure-supportability/per-vm-quota-requests__" L10: "}" L11: "}" L12: "--------------------------------------------------------------------------------_" L13: " " ```
🟢 ok **cl.ignition.luks**; Succeeded: azure (1) 🟢 ok **cl.ignition.misc.empty**; Succeeded: azure (2); Failed: azure (1)
                Diagnostic output for azure, run 1 ``` L1: " Error: _harness.go:646: Cluster failed starting machines: PUT https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kola-cluster-image-b4ba8baeca/providers/Mic?rosoft.Compute/virtualMachines/ci-4081.3.9-n-45fb5e28e0" L2: "--------------------------------------------------------------------------------" L3: "RESPONSE 409: 409 Conflict" L4: "ERROR CODE: OperationNotAllowed" L5: "--------------------------------------------------------------------------------" L6: "{" L7: "__error__: {" L8: "__code__: __OperationNotAllowed__," L9: "__message__: __Operation could not be completed as it results in exceeding approved standardDSv4Family Cores quota. Additional details - Deployment Model: Resource Manager, Location: northeurope, Curr?ent Limit: 65, Current Usage: 64, Additional Required: 2, (Minimum) New Limit Required: 66. Setup Alerts when Quota reaches threshold. Learn more at https://aka.ms/quotamonitoringalerting . Submit a r?equest for Quota increase at https://aka.ms/ProdportalCRP/#blade/Microsoft_Azure_Capacity/UsageAndQuota.ReactView/Parameters/%7B%22subscriptionId%22:%220e46bd28-a80f-4d3a-8200-d9eb8d80cb2e%22,%22comma?nd%22:%22openQuotaApprovalBlade%22,%22quotas%22:[%7B%22location%22:%22northeurope%22,%22providerId%22:%22Microsoft.Compute%22,%22resourceName%22:%22standardDSv4Family%22,%22quotaRequest%22:%7B%22prope?rties%22:%7B%22limit%22:66,%22unit%22:%22Count%22,%22name%22:%7B%22value%22:%22standardDSv4Family%22%7D%7D%7D%7D]%7D by specifying parameters listed in the ???Details??? section for deployment to succ?eed. Please read more about quota limits at https://docs.microsoft.com/en-us/azure/azure-supportability/per-vm-quota-requests__" L10: "}" L11: "}" L12: "--------------------------------------------------------------------------------_" L13: " " ```
🟢 ok **cl.ignition.symlink**; Succeeded: azure (2); Failed: azure (1)
                Diagnostic output for azure, run 1 ``` L1: " Error: _harness.go:646: Cluster failed starting machines: PUT https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kola-cluster-image-b4ba8baeca/providers/Mic?rosoft.Compute/virtualMachines/ci-4081.3.9-n-9f3868eea6" L2: "--------------------------------------------------------------------------------" L3: "RESPONSE 409: 409 Conflict" L4: "ERROR CODE: OperationNotAllowed" L5: "--------------------------------------------------------------------------------" L6: "{" L7: "__error__: {" L8: "__code__: __OperationNotAllowed__," L9: "__message__: __Operation could not be completed as it results in exceeding approved standardDSv4Family Cores quota. Additional details - Deployment Model: Resource Manager, Location: northeurope, Curr?ent Limit: 65, Current Usage: 64, Additional Required: 2, (Minimum) New Limit Required: 66. Setup Alerts when Quota reaches threshold. Learn more at https://aka.ms/quotamonitoringalerting . Submit a r?equest for Quota increase at https://aka.ms/ProdportalCRP/#blade/Microsoft_Azure_Capacity/UsageAndQuota.ReactView/Parameters/%7B%22subscriptionId%22:%220e46bd28-a80f-4d3a-8200-d9eb8d80cb2e%22,%22comma?nd%22:%22openQuotaApprovalBlade%22,%22quotas%22:[%7B%22location%22:%22northeurope%22,%22providerId%22:%22Microsoft.Compute%22,%22resourceName%22:%22standardDSv4Family%22,%22quotaRequest%22:%7B%22prope?rties%22:%7B%22limit%22:66,%22unit%22:%22Count%22,%22name%22:%7B%22value%22:%22standardDSv4Family%22%7D%7D%7D%7D]%7D by specifying parameters listed in the ???Details??? section for deployment to succ?eed. Please read more about quota limits at https://docs.microsoft.com/en-us/azure/azure-supportability/per-vm-quota-requests__" L10: "}" L11: "}" L12: "--------------------------------------------------------------------------------_" L13: " " ```
🟢 ok **cl.ignition.translation**; Succeeded: azure (1) 🟢 ok **cl.ignition.v1.btrfsroot**; Succeeded: azure (1) 🟢 ok **cl.ignition.v1.ext4root**; Succeeded: azure (2); Failed: azure (1)
                Diagnostic output for azure, run 1 ``` L1: " Error: _harness.go:646: Cluster failed starting machines: PUT https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kola-cluster-image-b4ba8baeca/providers/Mic?rosoft.Compute/virtualMachines/ci-4081.3.9-n-3a38ec7a9c" L2: "--------------------------------------------------------------------------------" L3: "RESPONSE 409: 409 Conflict" L4: "ERROR CODE: OperationNotAllowed" L5: "--------------------------------------------------------------------------------" L6: "{" L7: "__error__: {" L8: "__code__: __OperationNotAllowed__," L9: "__message__: __Operation could not be completed as it results in exceeding approved standardDSv4Family Cores quota. Additional details - Deployment Model: Resource Manager, Location: northeurope, Curr?ent Limit: 65, Current Usage: 64, Additional Required: 2, (Minimum) New Limit Required: 66. Setup Alerts when Quota reaches threshold. Learn more at https://aka.ms/quotamonitoringalerting . Submit a r?equest for Quota increase at https://aka.ms/ProdportalCRP/#blade/Microsoft_Azure_Capacity/UsageAndQuota.ReactView/Parameters/%7B%22subscriptionId%22:%220e46bd28-a80f-4d3a-8200-d9eb8d80cb2e%22,%22comma?nd%22:%22openQuotaApprovalBlade%22,%22quotas%22:[%7B%22location%22:%22northeurope%22,%22providerId%22:%22Microsoft.Compute%22,%22resourceName%22:%22standardDSv4Family%22,%22quotaRequest%22:%7B%22prope?rties%22:%7B%22limit%22:66,%22unit%22:%22Count%22,%22name%22:%7B%22value%22:%22standardDSv4Family%22%7D%7D%7D%7D]%7D by specifying parameters listed in the ???Details??? section for deployment to succ?eed. Please read more about quota limits at https://docs.microsoft.com/en-us/azure/azure-supportability/per-vm-quota-requests__" L10: "}" L11: "}" L12: "--------------------------------------------------------------------------------_" L13: " " ```
🟢 ok **cl.ignition.v1.groups**; Succeeded: azure (1) 🟢 ok **cl.ignition.v1.noop**; Succeeded: azure (2); Failed: azure (1)
                Diagnostic output for azure, run 1 ``` L1: " Error: _harness.go:646: Cluster failed starting machines: PUT https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kola-cluster-image-b4ba8baeca/providers/Mic?rosoft.Compute/virtualMachines/ci-4081.3.9-n-f567714ebc" L2: "--------------------------------------------------------------------------------" L3: "RESPONSE 409: 409 Conflict" L4: "ERROR CODE: OperationNotAllowed" L5: "--------------------------------------------------------------------------------" L6: "{" L7: "__error__: {" L8: "__code__: __OperationNotAllowed__," L9: "__message__: __Operation could not be completed as it results in exceeding approved standardDSv4Family Cores quota. Additional details - Deployment Model: Resource Manager, Location: northeurope, Curr?ent Limit: 65, Current Usage: 64, Additional Required: 2, (Minimum) New Limit Required: 66. Setup Alerts when Quota reaches threshold. Learn more at https://aka.ms/quotamonitoringalerting . Submit a r?equest for Quota increase at https://aka.ms/ProdportalCRP/#blade/Microsoft_Azure_Capacity/UsageAndQuota.ReactView/Parameters/%7B%22subscriptionId%22:%220e46bd28-a80f-4d3a-8200-d9eb8d80cb2e%22,%22comma?nd%22:%22openQuotaApprovalBlade%22,%22quotas%22:[%7B%22location%22:%22northeurope%22,%22providerId%22:%22Microsoft.Compute%22,%22resourceName%22:%22standardDSv4Family%22,%22quotaRequest%22:%7B%22prope?rties%22:%7B%22limit%22:66,%22unit%22:%22Count%22,%22name%22:%7B%22value%22:%22standardDSv4Family%22%7D%7D%7D%7D]%7D by specifying parameters listed in the ???Details??? section for deployment to succ?eed. Please read more about quota limits at https://docs.microsoft.com/en-us/azure/azure-supportability/per-vm-quota-requests__" L10: "}" L11: "}" L12: "--------------------------------------------------------------------------------_" L13: " " ```
🟢 ok **cl.ignition.v1.once**; Succeeded: azure (1) 🟢 ok **cl.ignition.v1.users**; Succeeded: azure (1) 🟢 ok **cl.ignition.v1.xfsroot**; Succeeded: azure (1) 🟢 ok **cl.ignition.v2.btrfsroot**; Succeeded: azure (1) 🟢 ok **cl.ignition.v2.ext4root**; Succeeded: azure (2); Failed: azure (1)
                Diagnostic output for azure, run 1 ``` L1: " Error: _harness.go:646: Cluster failed starting machines: PUT https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kola-cluster-image-b4ba8baeca/providers/Mic?rosoft.Compute/virtualMachines/ci-4081.3.9-n-50b47ff971" L2: "--------------------------------------------------------------------------------" L3: "RESPONSE 409: 409 Conflict" L4: "ERROR CODE: OperationNotAllowed" L5: "--------------------------------------------------------------------------------" L6: "{" L7: "__error__: {" L8: "__code__: __OperationNotAllowed__," L9: "__message__: __Operation could not be completed as it results in exceeding approved standardDSv4Family Cores quota. Additional details - Deployment Model: Resource Manager, Location: northeurope, Curr?ent Limit: 65, Current Usage: 64, Additional Required: 2, (Minimum) New Limit Required: 66. Setup Alerts when Quota reaches threshold. Learn more at https://aka.ms/quotamonitoringalerting . Submit a r?equest for Quota increase at https://aka.ms/ProdportalCRP/#blade/Microsoft_Azure_Capacity/UsageAndQuota.ReactView/Parameters/%7B%22subscriptionId%22:%220e46bd28-a80f-4d3a-8200-d9eb8d80cb2e%22,%22comma?nd%22:%22openQuotaApprovalBlade%22,%22quotas%22:[%7B%22location%22:%22northeurope%22,%22providerId%22:%22Microsoft.Compute%22,%22resourceName%22:%22standardDSv4Family%22,%22quotaRequest%22:%7B%22prope?rties%22:%7B%22limit%22:66,%22unit%22:%22Count%22,%22name%22:%7B%22value%22:%22standardDSv4Family%22%7D%7D%7D%7D]%7D by specifying parameters listed in the ???Details??? section for deployment to succ?eed. Please read more about quota limits at https://docs.microsoft.com/en-us/azure/azure-supportability/per-vm-quota-requests__" L10: "}" L11: "}" L12: "--------------------------------------------------------------------------------_" L13: " " ```
🟢 ok **cl.ignition.v2.noop**; Succeeded: azure (1) 🟢 ok **cl.ignition.v2.users**; Succeeded: azure (1) 🟢 ok **cl.ignition.v2.xfsroot**; Succeeded: azure (1) 🟢 ok **cl.ignition.v2_1.ext4checkexisting**; Succeeded: azure (2); Failed: azure (1)
                Diagnostic output for azure, run 1 ``` L1: " Error: _harness.go:646: Cluster failed starting machines: PUT https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kola-cluster-image-b4ba8baeca/providers/Mic?rosoft.Compute/virtualMachines/ci-4081.3.9-n-8754a44e19" L2: "--------------------------------------------------------------------------------" L3: "RESPONSE 409: 409 Conflict" L4: "ERROR CODE: OperationNotAllowed" L5: "--------------------------------------------------------------------------------" L6: "{" L7: "__error__: {" L8: "__code__: __OperationNotAllowed__," L9: "__message__: __Operation could not be completed as it results in exceeding approved standardDSv4Family Cores quota. Additional details - Deployment Model: Resource Manager, Location: northeurope, Curr?ent Limit: 65, Current Usage: 64, Additional Required: 2, (Minimum) New Limit Required: 66. Setup Alerts when Quota reaches threshold. Learn more at https://aka.ms/quotamonitoringalerting . Submit a r?equest for Quota increase at https://aka.ms/ProdportalCRP/#blade/Microsoft_Azure_Capacity/UsageAndQuota.ReactView/Parameters/%7B%22subscriptionId%22:%220e46bd28-a80f-4d3a-8200-d9eb8d80cb2e%22,%22comma?nd%22:%22openQuotaApprovalBlade%22,%22quotas%22:[%7B%22location%22:%22northeurope%22,%22providerId%22:%22Microsoft.Compute%22,%22resourceName%22:%22standardDSv4Family%22,%22quotaRequest%22:%7B%22prope?rties%22:%7B%22limit%22:66,%22unit%22:%22Count%22,%22name%22:%7B%22value%22:%22standardDSv4Family%22%7D%7D%7D%7D]%7D by specifying parameters listed in the ???Details??? section for deployment to succ?eed. Please read more about quota limits at https://docs.microsoft.com/en-us/azure/azure-supportability/per-vm-quota-requests__" L10: "}" L11: "}" L12: "--------------------------------------------------------------------------------_" L13: " " ```
🟢 ok **cl.ignition.v2_1.swap**; Succeeded: azure (1) 🟢 ok **cl.ignition.v2_1.vfat**; Succeeded: azure (1) 🟢 ok **cl.internet**; Succeeded: azure (3); Failed: azure (1, 2)
                Diagnostic output for azure, run 2 ``` L1: " Error: _harness.go:646: Cluster failed starting machines: PUT https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kola-cluster-image-922124aacf/providers/Mic?rosoft.Compute/virtualMachines/ci-4081.3.9-n-b36d68c97e" L2: "--------------------------------------------------------------------------------" L3: "RESPONSE 409: 409 Conflict" L4: "ERROR CODE: OperationNotAllowed" L5: "--------------------------------------------------------------------------------" L6: "{" L7: "__error__: {" L8: "__code__: __OperationNotAllowed__," L9: "__message__: __Operation could not be completed as it results in exceeding approved standardDSv4Family Cores quota. Additional details - Deployment Model: Resource Manager, Location: northeurope, Curr?ent Limit: 65, Current Usage: 64, Additional Required: 2, (Minimum) New Limit Required: 66. Setup Alerts when Quota reaches threshold. Learn more at https://aka.ms/quotamonitoringalerting . Submit a r?equest for Quota increase at https://aka.ms/ProdportalCRP/#blade/Microsoft_Azure_Capacity/UsageAndQuota.ReactView/Parameters/%7B%22subscriptionId%22:%220e46bd28-a80f-4d3a-8200-d9eb8d80cb2e%22,%22comma?nd%22:%22openQuotaApprovalBlade%22,%22quotas%22:[%7B%22location%22:%22northeurope%22,%22providerId%22:%22Microsoft.Compute%22,%22resourceName%22:%22standardDSv4Family%22,%22quotaRequest%22:%7B%22prope?rties%22:%7B%22limit%22:66,%22unit%22:%22Count%22,%22name%22:%7B%22value%22:%22standardDSv4Family%22%7D%7D%7D%7D]%7D by specifying parameters listed in the ???Details??? section for deployment to succ?eed. Please read more about quota limits at https://docs.microsoft.com/en-us/azure/azure-supportability/per-vm-quota-requests__" L10: "}" L11: "}" L12: "--------------------------------------------------------------------------------_" L13: " " ```                 Diagnostic output for azure, run 1 ``` L1: " Error: _harness.go:646: Cluster failed starting machines: creating public ip: PUT https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kola-cluster-image-b4ba?8baeca/providers/Microsoft.Network/publicIPAddresses/ip-a1c5101925" L2: "--------------------------------------------------------------------------------" L3: "RESPONSE 409: 409 Conflict" L4: "ERROR CODE: ResourceGroupBeingDeleted" L5: "--------------------------------------------------------------------------------" L6: "{" L7: "__error__: {" L8: "__code__: __ResourceGroupBeingDeleted__," L9: "__message__: __The resource group _kola-cluster-image-b4ba8baeca_ is in deprovisioning state and cannot perform this operation.__" L10: "}" L11: "}" L12: "--------------------------------------------------------------------------------_" L13: " " ```
🟢 ok **cl.locksmith.cluster**; Succeeded: azure (2); Failed: azure (1)
                Diagnostic output for azure, run 1 ``` L1: " Error: _locksmith.go:184: [0] ssh unreachable or system not ready: context deadline exceeded [1] ssh unreachable or system not ready: context deadline exceeded [2] ssh unreachable or system not ready:? context deadline exceeded_" L2: " " ```
🟢 ok **cl.metadata.azure**; Succeeded: azure (2); Failed: azure (1)
                Diagnostic output for azure, run 1 ``` L1: " Error: _harness.go:646: Cluster failed starting machines: PUT https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kola-cluster-image-b4ba8baeca/providers/Mic?rosoft.Compute/virtualMachines/ci-4081.3.9-n-8546026117" L2: "--------------------------------------------------------------------------------" L3: "RESPONSE 409: 409 Conflict" L4: "ERROR CODE: OperationNotAllowed" L5: "--------------------------------------------------------------------------------" L6: "{" L7: "__error__: {" L8: "__code__: __OperationNotAllowed__," L9: "__message__: __Operation could not be completed as it results in exceeding approved standardDSv4Family Cores quota. Additional details - Deployment Model: Resource Manager, Location: northeurope, Curr?ent Limit: 65, Current Usage: 64, Additional Required: 2, (Minimum) New Limit Required: 66. Setup Alerts when Quota reaches threshold. Learn more at https://aka.ms/quotamonitoringalerting . Submit a r?equest for Quota increase at https://aka.ms/ProdportalCRP/#blade/Microsoft_Azure_Capacity/UsageAndQuota.ReactView/Parameters/%7B%22subscriptionId%22:%220e46bd28-a80f-4d3a-8200-d9eb8d80cb2e%22,%22comma?nd%22:%22openQuotaApprovalBlade%22,%22quotas%22:[%7B%22location%22:%22northeurope%22,%22providerId%22:%22Microsoft.Compute%22,%22resourceName%22:%22standardDSv4Family%22,%22quotaRequest%22:%7B%22prope?rties%22:%7B%22limit%22:66,%22unit%22:%22Count%22,%22name%22:%7B%22value%22:%22standardDSv4Family%22%7D%7D%7D%7D]%7D by specifying parameters listed in the ???Details??? section for deployment to succ?eed. Please read more about quota limits at https://docs.microsoft.com/en-us/azure/azure-supportability/per-vm-quota-requests__" L10: "}" L11: "}" L12: "--------------------------------------------------------------------------------_" L13: " " ```
🟢 ok **cl.network.initramfs.second-boot**; Succeeded: azure (1) ❌ not ok **cl.network.iptables**; Failed: azure (1, 2, 3, 4, 5)
                Diagnostic output for azure, run 5 ``` L1: " Error: _cluster.go:152: + sudo nft --json list ruleset | jq _.nftables[] | select(.rule) | .rule.expr[0].match.right_" L2: "cluster.go:156: cmd sudo nft --json list ruleset | jq _.nftables[] | select(.rule) | .rule.expr[0].match.right_ did not output 80_" L3: " " ```                 Diagnostic output for azure, run 4 ``` L1: " Error: _cluster.go:152: + sudo nft --json list ruleset | jq _.nftables[] | select(.rule) | .rule.expr[0].match.right_" L2: "cluster.go:156: cmd sudo nft --json list ruleset | jq _.nftables[] | select(.rule) | .rule.expr[0].match.right_ did not output 80_" L3: " " ```                 Diagnostic output for azure, run 3 ``` L1: " Error: _cluster.go:152: + sudo nft --json list ruleset | jq _.nftables[] | select(.rule) | .rule.expr[0].match.right_" L2: "cluster.go:156: cmd sudo nft --json list ruleset | jq _.nftables[] | select(.rule) | .rule.expr[0].match.right_ did not output 80_" L3: " " ```                 Diagnostic output for azure, run 2 ``` L1: " Error: _cluster.go:152: + sudo nft --json list ruleset | jq _.nftables[] | select(.rule) | .rule.expr[0].match.right_" L2: "cluster.go:156: cmd sudo nft --json list ruleset | jq _.nftables[] | select(.rule) | .rule.expr[0].match.right_ did not output 80_" L3: " " ```                 Diagnostic output for azure, run 1 ``` L1: " Error: _cluster.go:152: + sudo nft --json list ruleset | jq _.nftables[] | select(.rule) | .rule.expr[0].match.right_" L2: "cluster.go:156: cmd sudo nft --json list ruleset | jq _.nftables[] | select(.rule) | .rule.expr[0].match.right_ did not output 80_" L3: " " ```
🟢 ok **cl.network.wireguard**; Succeeded: azure (2); Failed: azure (1)
                Diagnostic output for azure, run 1 ``` L1: " Error: _harness.go:646: Cluster failed starting machines: PollUntilDone(ci-4081.3.9-n-48a5ce6ce5): GET https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/providers/Microso?ft.Compute/locations/northeurope/operations/f2fbbc82-5e26-4998-af9f-38ac0021dea6" L2: "--------------------------------------------------------------------------------" L3: "RESPONSE 200: 200 OK" L4: "ERROR CODE: OperationPreempted" L5: "--------------------------------------------------------------------------------" L6: "{" L7: "__startTime__: __2026-07-24T23:44:19.6303073+00:00__," L8: "__endTime__: __2026-07-24T23:44:49.7612685+00:00__," L9: "__status__: __Canceled__," L10: "__error__: {" L11: "__code__: __OperationPreempted__," L12: "__message__: __Operation execution has been preempted by a more recent operation.__" L13: "}," L14: "__name__: __f2fbbc82-5e26-4998-af9f-38ac0021dea6__" L15: "}" L16: "--------------------------------------------------------------------------------_" L17: " " ```
🟢 ok **cl.osreset.ignition-rerun**; Succeeded: azure (1) 🟢 ok **cl.overlay.cleanup**; Succeeded: azure (2); Failed: azure (1)
                Diagnostic output for azure, run 1 ``` L1: " Error: _state.go:97: could not reboot: machine __ci-4081.3.9-n-c3c25c9f5b__ failed basic checks: some systemd units failed:" L2: "??? ldconfig.service loaded failed failed Rebuild Dynamic Linker Cache" L3: "status: " L4: "journal:-- No entries --_" L5: " " ```
🟢 ok **cl.swap_activation**; Succeeded: azure (2); Failed: azure (1)
                Diagnostic output for azure, run 1 ``` L1: " Error: _harness.go:646: Cluster failed starting machines: PUT https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kola-cluster-image-b4ba8baeca/providers/Mic?rosoft.Compute/virtualMachines/ci-4081.3.9-n-30c5662a47" L2: "--------------------------------------------------------------------------------" L3: "RESPONSE 409: 409 Conflict" L4: "ERROR CODE: OperationNotAllowed" L5: "--------------------------------------------------------------------------------" L6: "{" L7: "__error__: {" L8: "__code__: __OperationNotAllowed__," L9: "__message__: __Operation could not be completed as it results in exceeding approved standardDSv4Family Cores quota. Additional details - Deployment Model: Resource Manager, Location: northeurope, Curr?ent Limit: 65, Current Usage: 64, Additional Required: 2, (Minimum) New Limit Required: 66. Setup Alerts when Quota reaches threshold. Learn more at https://aka.ms/quotamonitoringalerting . Submit a r?equest for Quota increase at https://aka.ms/ProdportalCRP/#blade/Microsoft_Azure_Capacity/UsageAndQuota.ReactView/Parameters/%7B%22subscriptionId%22:%220e46bd28-a80f-4d3a-8200-d9eb8d80cb2e%22,%22comma?nd%22:%22openQuotaApprovalBlade%22,%22quotas%22:[%7B%22location%22:%22northeurope%22,%22providerId%22:%22Microsoft.Compute%22,%22resourceName%22:%22standardDSv4Family%22,%22quotaRequest%22:%7B%22prope?rties%22:%7B%22limit%22:66,%22unit%22:%22Count%22,%22name%22:%7B%22value%22:%22standardDSv4Family%22%7D%7D%7D%7D]%7D by specifying parameters listed in the ???Details??? section for deployment to succ?eed. Please read more about quota limits at https://docs.microsoft.com/en-us/azure/azure-supportability/per-vm-quota-requests__" L10: "}" L11: "}" L12: "--------------------------------------------------------------------------------_" L13: " " ```
🟢 ok **cl.toolbox.dnf-install**; Succeeded: azure (1) 🟢 ok **cl.update.badverity**; Succeeded: azure (2); Failed: azure (1)
                Diagnostic output for azure, run 1 ``` L1: " Error: _update.go:168: ssh unreachable or system not ready: failure checking if machine is running: systemctl is-system-running returned stdout: ____, stderr: ____, err: dial tcp 23.100.51.33:22: i/o ?timeout, systemctl list-jobs returned stdout: ____, stderr: ____, err: dial tcp 23.100.51.33:22: i/o timeout_" L2: " " ```
🟢 ok **cl.update.reboot**; Succeeded: azure (1) 🟢 ok **cl.users.shells**; Succeeded: azure (1) 🟢 ok **cl.verity**; Succeeded: azure (2); Failed: azure (1)
                Diagnostic output for azure, run 1 ``` L1: " Error: _harness.go:646: Cluster failed starting machines: PUT https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kola-cluster-image-b4ba8baeca/providers/Mic?rosoft.Compute/virtualMachines/ci-4081.3.9-n-1a84f62afa" L2: "--------------------------------------------------------------------------------" L3: "RESPONSE 409: 409 Conflict" L4: "ERROR CODE: OperationNotAllowed" L5: "--------------------------------------------------------------------------------" L6: "{" L7: "__error__: {" L8: "__code__: __OperationNotAllowed__," L9: "__message__: __Operation could not be completed as it results in exceeding approved standardDSv4Family Cores quota. Additional details - Deployment Model: Resource Manager, Location: northeurope, Curr?ent Limit: 65, Current Usage: 64, Additional Required: 2, (Minimum) New Limit Required: 66. Setup Alerts when Quota reaches threshold. Learn more at https://aka.ms/quotamonitoringalerting . Submit a r?equest for Quota increase at https://aka.ms/ProdportalCRP/#blade/Microsoft_Azure_Capacity/UsageAndQuota.ReactView/Parameters/%7B%22subscriptionId%22:%220e46bd28-a80f-4d3a-8200-d9eb8d80cb2e%22,%22comma?nd%22:%22openQuotaApprovalBlade%22,%22quotas%22:[%7B%22location%22:%22northeurope%22,%22providerId%22:%22Microsoft.Compute%22,%22resourceName%22:%22standardDSv4Family%22,%22quotaRequest%22:%7B%22prope?rties%22:%7B%22limit%22:66,%22unit%22:%22Count%22,%22name%22:%7B%22value%22:%22standardDSv4Family%22%7D%7D%7D%7D]%7D by specifying parameters listed in the ???Details??? section for deployment to succ?eed. Please read more about quota limits at https://docs.microsoft.com/en-us/azure/azure-supportability/per-vm-quota-requests__" L10: "}" L11: "}" L12: "--------------------------------------------------------------------------------_" L13: " " ```
🟢 ok **coreos.auth.verify**; Succeeded: azure (1) 🟢 ok **coreos.ignition.groups**; Succeeded: azure (1) 🟢 ok **coreos.ignition.once**; Succeeded: azure (1) 🟢 ok **coreos.ignition.resource.local**; Succeeded: azure (2); Failed: azure (1)
                Diagnostic output for azure, run 1 ``` L1: " Error: _harness.go:646: Cluster failed starting machines: PUT https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kola-cluster-image-b4ba8baeca/providers/Mic?rosoft.Compute/virtualMachines/ci-4081.3.9-n-810781086b" L2: "--------------------------------------------------------------------------------" L3: "RESPONSE 409: 409 Conflict" L4: "ERROR CODE: OperationNotAllowed" L5: "--------------------------------------------------------------------------------" L6: "{" L7: "__error__: {" L8: "__code__: __OperationNotAllowed__," L9: "__message__: __Operation could not be completed as it results in exceeding approved standardDSv4Family Cores quota. Additional details - Deployment Model: Resource Manager, Location: northeurope, Curr?ent Limit: 65, Current Usage: 64, Additional Required: 2, (Minimum) New Limit Required: 66. Setup Alerts when Quota reaches threshold. Learn more at https://aka.ms/quotamonitoringalerting . Submit a r?equest for Quota increase at https://aka.ms/ProdportalCRP/#blade/Microsoft_Azure_Capacity/UsageAndQuota.ReactView/Parameters/%7B%22subscriptionId%22:%220e46bd28-a80f-4d3a-8200-d9eb8d80cb2e%22,%22comma?nd%22:%22openQuotaApprovalBlade%22,%22quotas%22:[%7B%22location%22:%22northeurope%22,%22providerId%22:%22Microsoft.Compute%22,%22resourceName%22:%22standardDSv4Family%22,%22quotaRequest%22:%7B%22prope?rties%22:%7B%22limit%22:66,%22unit%22:%22Count%22,%22name%22:%7B%22value%22:%22standardDSv4Family%22%7D%7D%7D%7D]%7D by specifying parameters listed in the ???Details??? section for deployment to succ?eed. Please read more about quota limits at https://docs.microsoft.com/en-us/azure/azure-supportability/per-vm-quota-requests__" L10: "}" L11: "}" L12: "--------------------------------------------------------------------------------_" L13: " " ```
🟢 ok **coreos.ignition.resource.remote**; Succeeded: azure (2); Failed: azure (1)
                Diagnostic output for azure, run 1 ``` L1: " Error: _harness.go:646: Cluster failed starting machines: PUT https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kola-cluster-image-b4ba8baeca/providers/Mic?rosoft.Compute/virtualMachines/ci-4081.3.9-n-07e8f4f7fd" L2: "--------------------------------------------------------------------------------" L3: "RESPONSE 409: 409 Conflict" L4: "ERROR CODE: OperationNotAllowed" L5: "--------------------------------------------------------------------------------" L6: "{" L7: "__error__: {" L8: "__code__: __OperationNotAllowed__," L9: "__message__: __Operation could not be completed as it results in exceeding approved standardDSv4Family Cores quota. Additional details - Deployment Model: Resource Manager, Location: northeurope, Curr?ent Limit: 65, Current Usage: 64, Additional Required: 2, (Minimum) New Limit Required: 66. Setup Alerts when Quota reaches threshold. Learn more at https://aka.ms/quotamonitoringalerting . Submit a r?equest for Quota increase at https://aka.ms/ProdportalCRP/#blade/Microsoft_Azure_Capacity/UsageAndQuota.ReactView/Parameters/%7B%22subscriptionId%22:%220e46bd28-a80f-4d3a-8200-d9eb8d80cb2e%22,%22comma?nd%22:%22openQuotaApprovalBlade%22,%22quotas%22:[%7B%22location%22:%22northeurope%22,%22providerId%22:%22Microsoft.Compute%22,%22resourceName%22:%22standardDSv4Family%22,%22quotaRequest%22:%7B%22prope?rties%22:%7B%22limit%22:66,%22unit%22:%22Count%22,%22name%22:%7B%22value%22:%22standardDSv4Family%22%7D%7D%7D%7D]%7D by specifying parameters listed in the ???Details??? section for deployment to succ?eed. Please read more about quota limits at https://docs.microsoft.com/en-us/azure/azure-supportability/per-vm-quota-requests__" L10: "}" L11: "}" L12: "--------------------------------------------------------------------------------_" L13: " " ```
🟢 ok **coreos.ignition.security.tls**; Succeeded: azure (1) ❌ not ok **coreos.ignition.ssh.key**; Failed: azure (1, 2, 3, 4, 5)
                Diagnostic output for azure, run 5 ``` L1: " Error: _harness.go:646: Cluster failed starting machines: machine __ci-4081.3.9-n-21274b3070__ failed to start: ssh journalctl failed: time limit exceeded: ssh: handshake failed: ssh: unable to authen?ticate, attempted methods [none publickey], no supported methods remain_" L2: " " ```                 Diagnostic output for azure, run 4 ``` L1: " Error: _harness.go:646: Cluster failed starting machines: machine __ci-4081.3.9-n-a337149f0a__ failed to start: ssh journalctl failed: time limit exceeded: dial tcp 40.112.76.124:22: i/o timeout_" L2: " " ```                 Diagnostic output for azure, run 3 ``` L1: " Error: _harness.go:646: Cluster failed starting machines: machine __ci-4081.3.9-n-11473f5f07__ failed to start: ssh journalctl failed: time limit exceeded: ssh: handshake failed: ssh: unable to authen?ticate, attempted methods [none publickey], no supported methods remain_" L2: " " ```                 Diagnostic output for azure, run 2 ``` L1: " Error: _harness.go:646: Cluster failed starting machines: machine __ci-4081.3.9-n-4481bf2b30__ failed to start: ssh journalctl failed: time limit exceeded: ssh: handshake failed: ssh: unable to authen?ticate, attempted methods [none publickey], no supported methods remain_" L2: " " ```                 Diagnostic output for azure, run 1 ``` L1: " Error: _harness.go:646: Cluster failed starting machines: machine __ci-4081.3.9-n-550cf4fa79__ failed to start: ssh journalctl failed: time limit exceeded: dial tcp 20.223.220.192:22: i/o timeout_" L2: " " ```
🟢 ok **coreos.ignition.systemd.enable-service**; Succeeded: azure (2); Failed: azure (1)
                Diagnostic output for azure, run 1 ``` L1: " Error: _harness.go:646: Cluster failed starting machines: PollUntilDone(ci-4081.3.9-n-5e03a42476): GET https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/providers/Microso?ft.Compute/locations/northeurope/operations/48196cb8-347d-45e4-a7c7-66be86d4adce" L2: "--------------------------------------------------------------------------------" L3: "RESPONSE 200: 200 OK" L4: "ERROR CODE: OperationPreempted" L5: "--------------------------------------------------------------------------------" L6: "{" L7: "__startTime__: __2026-07-24T23:43:24.1028624+00:00__," L8: "__endTime__: __2026-07-24T23:44:56.2544875+00:00__," L9: "__status__: __Canceled__," L10: "__error__: {" L11: "__code__: __OperationPreempted__," L12: "__message__: __Operation execution has been preempted by a more recent operation.__" L13: "}," L14: "__name__: __48196cb8-347d-45e4-a7c7-66be86d4adce__" L15: "}" L16: "--------------------------------------------------------------------------------_" L17: " " ```
🟢 ok **coreos.locksmith.reboot**; Succeeded: azure (2); Failed: azure (1)
                Diagnostic output for azure, run 1 ``` L1: " Error: _harness.go:646: Cluster failed starting machines: PollUntilDone(ci-4081.3.9-n-b3db7dc9be): GET https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/providers/Microso?ft.Compute/locations/northeurope/operations/465949d4-ccf4-49bf-8acb-07e3531891cb" L2: "--------------------------------------------------------------------------------" L3: "RESPONSE 200: 200 OK" L4: "ERROR CODE: OperationPreempted" L5: "--------------------------------------------------------------------------------" L6: "{" L7: "__startTime__: __2026-07-24T23:43:46.4072559+00:00__," L8: "__endTime__: __2026-07-24T23:44:56.6085682+00:00__," L9: "__status__: __Canceled__," L10: "__error__: {" L11: "__code__: __OperationPreempted__," L12: "__message__: __Operation execution has been preempted by a more recent operation.__" L13: "}," L14: "__name__: __465949d4-ccf4-49bf-8acb-07e3531891cb__" L15: "}" L16: "--------------------------------------------------------------------------------_" L17: " " ```
🟢 ok **coreos.locksmith.tls**; Succeeded: azure (1) 🟢 ok **coreos.selinux.boolean**; Succeeded: azure (2); Failed: azure (1)
                Diagnostic output for azure, run 1 ``` L1: " Error: _selinux.go:115: failed to reboot machine: machine __ci-4081.3.9-n-c9d9c908f6__ failed to start: ssh journalctl failed: time limit exceeded: dial tcp 20.166.16.201:22: i/o timeout_" L2: " " ```
🟢 ok **coreos.selinux.enforce**; Succeeded: azure (1) 🟢 ok **coreos.tls.fetch-urls**; Succeeded: azure (1) 🟢 ok **coreos.update.badusr**; Succeeded: azure (2); Failed: azure (1)
                Diagnostic output for azure, run 1 ``` L1: " Error: _harness.go:646: Cluster failed starting machines: PollUntilDone(ci-4081.3.9-n-be98e2219e): GET https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/providers/Microso?ft.Compute/locations/northeurope/operations/7c266008-93dd-48ac-8fa8-c42743fd4a4c" L2: "--------------------------------------------------------------------------------" L3: "RESPONSE 200: 200 OK" L4: "ERROR CODE: OperationPreempted" L5: "--------------------------------------------------------------------------------" L6: "{" L7: "__startTime__: __2026-07-24T23:43:45.975776+00:00__," L8: "__endTime__: __2026-07-24T23:44:56.2596252+00:00__," L9: "__status__: __Canceled__," L10: "__error__: {" L11: "__code__: __OperationPreempted__," L12: "__message__: __Operation execution has been preempted by a more recent operation.__" L13: "}," L14: "__name__: __7c266008-93dd-48ac-8fa8-c42743fd4a4c__" L15: "}" L16: "--------------------------------------------------------------------------------_" L17: " " ```
🟢 ok **docker.base**; Succeeded: azure (1) 🟢 ok **docker.btrfs-storage**; Succeeded: azure (1) 🟢 ok **docker.containerd-restart**; Succeeded: azure (2); Failed: azure (1)
                Diagnostic output for azure, run 1 ``` L1: " Error: _harness.go:646: Cluster failed starting machines: PollUntilDone(ci-4081.3.9-n-812ac0ee10): GET https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/providers/Microso?ft.Compute/locations/northeurope/operations/45a59e58-5a1e-4fc1-a4c0-762ee1cf36f5" L2: "--------------------------------------------------------------------------------" L3: "RESPONSE 200: 200 OK" L4: "ERROR CODE: OperationPreempted" L5: "--------------------------------------------------------------------------------" L6: "{" L7: "__startTime__: __2026-07-24T23:44:26.1697488+00:00__," L8: "__endTime__: __2026-07-24T23:44:49.8738835+00:00__," L9: "__status__: __Canceled__," L10: "__error__: {" L11: "__code__: __OperationPreempted__," L12: "__message__: __Operation execution has been preempted by a more recent operation.__" L13: "}," L14: "__name__: __45a59e58-5a1e-4fc1-a4c0-762ee1cf36f5__" L15: "}" L16: "--------------------------------------------------------------------------------_" L17: " " ```
🟢 ok **docker.enable-service.sysext**; Succeeded: azure (2); Failed: azure (1)
                Diagnostic output for azure, run 1 ``` L1: " Error: _harness.go:646: Cluster failed starting machines: PUT https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kola-cluster-image-b4ba8baeca/providers/Mic?rosoft.Compute/virtualMachines/ci-4081.3.9-n-4e91a67da3" L2: "--------------------------------------------------------------------------------" L3: "RESPONSE 409: 409 Conflict" L4: "ERROR CODE: OperationNotAllowed" L5: "--------------------------------------------------------------------------------" L6: "{" L7: "__error__: {" L8: "__code__: __OperationNotAllowed__," L9: "__message__: __Operation could not be completed as it results in exceeding approved standardDSv4Family Cores quota. Additional details - Deployment Model: Resource Manager, Location: northeurope, Curr?ent Limit: 65, Current Usage: 64, Additional Required: 2, (Minimum) New Limit Required: 66. Setup Alerts when Quota reaches threshold. Learn more at https://aka.ms/quotamonitoringalerting . Submit a r?equest for Quota increase at https://aka.ms/ProdportalCRP/#blade/Microsoft_Azure_Capacity/UsageAndQuota.ReactView/Parameters/%7B%22subscriptionId%22:%220e46bd28-a80f-4d3a-8200-d9eb8d80cb2e%22,%22comma?nd%22:%22openQuotaApprovalBlade%22,%22quotas%22:[%7B%22location%22:%22northeurope%22,%22providerId%22:%22Microsoft.Compute%22,%22resourceName%22:%22standardDSv4Family%22,%22quotaRequest%22:%7B%22prope?rties%22:%7B%22limit%22:66,%22unit%22:%22Count%22,%22name%22:%7B%22value%22:%22standardDSv4Family%22%7D%7D%7D%7D]%7D by specifying parameters listed in the ???Details??? section for deployment to succ?eed. Please read more about quota limits at https://docs.microsoft.com/en-us/azure/azure-supportability/per-vm-quota-requests__" L10: "}" L11: "}" L12: "--------------------------------------------------------------------------------_" L13: " " ```
🟢 ok **docker.lib-coreos-dockerd-compat**; Succeeded: azure (2); Failed: azure (1)
                Diagnostic output for azure, run 1 ``` L1: " _docker.lib-coreos-dockerd-compat/networks-reliably (109.83s)" L2: "cluster.go:125: #1 [internal] load build definition from Dockerfile" L3: "cluster.go:125: #1 transferring dockerfile: 108B done" L4: "cluster.go:125: #1 DONE 0.0s" L5: "cluster.go:125: " L6: "cluster.go:125: #2 [internal] load .dockerignore" L7: "cluster.go:125: #2 transferring context: 2B done" L8: "cluster.go:125: #2 DONE 0.0s" L9: "cluster.go:125: " L10: "cluster.go:125: #3 [internal] load build context" L11: "cluster.go:125: #3 transferring context: 4.04MB 0.0s done" L12: "cluster.go:125: #3 DONE 0.1s" L13: "cluster.go:125: " L14: "cluster.go:125: #4 [1/1] COPY . /" L15: "cluster.go:125: #4 DONE 0.0s" L16: "cluster.go:125: " L17: "cluster.go:125: #5 exporting to image" L18: "cluster.go:125: #5 exporting layers" L19: "cluster.go:125: #5 exporting layers 0.1s done" L20: "cluster.go:125: #5 writing image sha256:3fdb5e2a1dec4bab24c1651b502132517e31d6f9d329d8eabf85e3c8632afb74 done" L21: "cluster.go:125: #5 naming to docker.io/library/ping done" L22: "cluster.go:125: #5 DONE 0.1s" L23: "cluster.go:145: __for i in $(seq 1 100); do_n_t_techo -n ___$i: ____n_t_tdocker run --rm ping sh -c _ping -i 0.2 172.17.0.1 -w 1 _/dev/null && echo PASS || echo FAIL__n_tdone__ failed: output 1: PASS" L24: "2: PASS" L25: "3: PASS" L26: "4: PASS" L27: "5: PASS" L28: "6: PASS" L29: "7: PASS" L30: "8: PASS" L31: "9: PASS" L32: "10: PASS" L33: "11: PASS" L34: "12: PASS" L35: "13: PASS" L36: "14: PASS" L37: "15: PASS" L38: "16: PASS" L39: "17: PASS" L40: "18: PASS" L41: "19: PASS" L42: "20: PASS" L43: "21: PASS" L44: "22: PASS" L45: "23: PASS" L46: "24: PASS" L47: "25: PASS" L48: "26: PASS" L49: "27: PASS" L50: "28: PASS" L51: "29: PASS" L52: "30: PASS" L53: "31: PASS" L54: "32: PASS" L55: "33: PASS" L56: "34: PASS" L57: "35: PASS" L58: "36: PASS" L59: "37: PASS" L60: "38: PASS" L61: "39: PASS" L62: "40: PASS" L63: "41: PASS" L64: "42: PASS" L65: "43: PASS" L66: "44: PASS" L67: "45: PASS" L68: "46: PASS" L69: "47: PASS" L70: "48: PASS" L71: "49: PASS" L72: "50: PASS" L73: "51: PASS" L74: "52: PASS" L75: "53: PASS" L76: "54: PASS" L77: "55: PASS" L78: "56: PASS" L79: "57: PASS" L80: "58: PASS" L81: "59: PASS" L82: "60: PASS" L83: "61: PASS" L84: "62: PASS" L85: "63: PASS" L86: "64: PASS" L87: "65: PASS" L88: "66: PASS" L89: "67: PASS" L90: "68: PASS" L91: "69: PASS" L92: "70: PASS" L93: "71:, status wait: remote command exited without exit status or exit signal" L94: "--- FAIL: docker.lib-coreos-dockerd-compat/user-no-caps (0.04s)" L95: "cluster.go:145: __tmpdir=$(mktemp -d); cd $tmpdir; echo -e ___FROM scratch__nCOPY . /___ _ Dockerfile;_n_t b=$(which capsh sh grep cat ls); libs=$(sudo ldd $b | grep -o /lib_[^ ]*_ | sort -u);_?n_t sudo rsync -av --relative --copy-links $b $libs ./;_n_t sudo docker build -t captest .__ failed: output , status ssh: handshake failed: read tcp 10.200.1.4:42860-_20.238.32.24:22: re?ad: connection reset by peer" L96: "--- FAIL: docker.lib-coreos-dockerd-compat/ownership (0.04s)" L97: "cluster.go:145: __docker run --name ownership ghcr.io/flatcar/nginx stat -c ___%u/%g___ /etc/shadow__ failed: output , status ssh: handshake failed: read tcp 10.200.1.4:42876-_20.238.32.24:22: read: c?onnection reset by peer_" L98: " " ```
🟢 ok **docker.network-openbsd-nc**; Succeeded: azure (2); Failed: azure (1)
                Diagnostic output for azure, run 1 ``` L1: " Error: _harness.go:646: Cluster failed starting machines: PUT https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kola-cluster-image-b4ba8baeca/providers/Mic?rosoft.Compute/virtualMachines/ci-4081.3.9-n-38719d2956" L2: "--------------------------------------------------------------------------------" L3: "RESPONSE 409: 409 Conflict" L4: "ERROR CODE: OperationNotAllowed" L5: "--------------------------------------------------------------------------------" L6: "{" L7: "__error__: {" L8: "__code__: __OperationNotAllowed__," L9: "__message__: __Operation could not be completed as it results in exceeding approved standardDSv4Family Cores quota. Additional details - Deployment Model: Resource Manager, Location: northeurope, Curr?ent Limit: 65, Current Usage: 64, Additional Required: 2, (Minimum) New Limit Required: 66. Setup Alerts when Quota reaches threshold. Learn more at https://aka.ms/quotamonitoringalerting . Submit a r?equest for Quota increase at https://aka.ms/ProdportalCRP/#blade/Microsoft_Azure_Capacity/UsageAndQuota.ReactView/Parameters/%7B%22subscriptionId%22:%220e46bd28-a80f-4d3a-8200-d9eb8d80cb2e%22,%22comma?nd%22:%22openQuotaApprovalBlade%22,%22quotas%22:[%7B%22location%22:%22northeurope%22,%22providerId%22:%22Microsoft.Compute%22,%22resourceName%22:%22standardDSv4Family%22,%22quotaRequest%22:%7B%22prope?rties%22:%7B%22limit%22:66,%22unit%22:%22Count%22,%22name%22:%7B%22value%22:%22standardDSv4Family%22%7D%7D%7D%7D]%7D by specifying parameters listed in the ???Details??? section for deployment to succ?eed. Please read more about quota limits at https://docs.microsoft.com/en-us/azure/azure-supportability/per-vm-quota-requests__" L10: "}" L11: "}" L12: "--------------------------------------------------------------------------------_" L13: " " ```
🟢 ok **docker.selinux**; Succeeded: azure (2); Failed: azure (1)
                Diagnostic output for azure, run 1 ``` L1: " Error: _harness.go:646: Cluster failed starting machines: waiting for machine to become active: GET https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kola-?cluster-image-b4ba8baeca/providers/Microsoft.Compute/virtualMachines/ci-4081.3.9-n-5890132e67" L2: "--------------------------------------------------------------------------------" L3: "RESPONSE 404: 404 Not Found" L4: "ERROR CODE: NotFound" L5: "--------------------------------------------------------------------------------" L6: "{" L7: "__error__: {" L8: "__code__: __NotFound__," L9: "__message__: __The entity was not found in this Azure location.__" L10: "}" L11: "}" L12: "--------------------------------------------------------------------------------_" L13: " " ```
🟢 ok **docker.userns**; Succeeded: azure (1) 🟢 ok **extra-test.[V1].cl.internet**; Succeeded: azure (1) 🟢 ok **kubeadm.v1.34.4.calico.base**; Succeeded: azure (2); Failed: azure (1)
                Diagnostic output for azure, run 1 ``` L1: " Error: _kubeadm.go:197: unable to setup cluster: unable to create etcd node: PUT https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kola-cluster-image-b4ba8?baeca/providers/Microsoft.Compute/virtualMachines/ci-4081.3.9-n-b4ca6a6d61" L2: "--------------------------------------------------------------------------------" L3: "RESPONSE 409: 409 Conflict" L4: "ERROR CODE: OperationNotAllowed" L5: "--------------------------------------------------------------------------------" L6: "{" L7: "__error__: {" L8: "__code__: __OperationNotAllowed__," L9: "__message__: __Operation could not be completed as it results in exceeding approved standardDSv4Family Cores quota. Additional details - Deployment Model: Resource Manager, Location: northeurope, Curr?ent Limit: 65, Current Usage: 64, Additional Required: 2, (Minimum) New Limit Required: 66. Setup Alerts when Quota reaches threshold. Learn more at https://aka.ms/quotamonitoringalerting . Submit a r?equest for Quota increase at https://aka.ms/ProdportalCRP/#blade/Microsoft_Azure_Capacity/UsageAndQuota.ReactView/Parameters/%7B%22subscriptionId%22:%220e46bd28-a80f-4d3a-8200-d9eb8d80cb2e%22,%22comma?nd%22:%22openQuotaApprovalBlade%22,%22quotas%22:[%7B%22location%22:%22northeurope%22,%22providerId%22:%22Microsoft.Compute%22,%22resourceName%22:%22standardDSv4Family%22,%22quotaRequest%22:%7B%22prope?rties%22:%7B%22limit%22:66,%22unit%22:%22Count%22,%22name%22:%7B%22value%22:%22standardDSv4Family%22%7D%7D%7D%7D]%7D by specifying parameters listed in the ???Details??? section for deployment to succ?eed. Please read more about quota limits at https://docs.microsoft.com/en-us/azure/azure-supportability/per-vm-quota-requests__" L10: "}" L11: "}" L12: "--------------------------------------------------------------------------------_" L13: " " ```
🟢 ok **kubeadm.v1.34.4.calico.cgroupv1.base**; Succeeded: azure (2); Failed: azure (1)
                Diagnostic output for azure, run 1 ``` L1: " Error: _cluster.go:125: I0724 23:41:16.931793 2511 version.go:260] remote version is much newer: v1.36.3; falling back to: stable-1.34" L2: "cluster.go:125: [config/images] Pulled registry.k8s.io/kube-apiserver:v1.34.10" L3: "cluster.go:125: [config/images] Pulled registry.k8s.io/kube-controller-manager:v1.34.10" L4: "cluster.go:125: [config/images] Pulled registry.k8s.io/kube-scheduler:v1.34.10" L5: "cluster.go:125: [config/images] Pulled registry.k8s.io/kube-proxy:v1.34.10" L6: "cluster.go:125: [config/images] Pulled registry.k8s.io/coredns/coredns:v1.12.1" L7: "cluster.go:125: [config/images] Pulled registry.k8s.io/pause:3.10.1" L8: "cluster.go:125: [config/images] Pulled registry.k8s.io/etcd:3.6.5-0" L9: "cluster.go:125: I0724 23:41:29.137450 2731 version.go:260] remote version is much newer: v1.36.3; falling back to: stable-1.34" L10: "cluster.go:125: [init] Using Kubernetes version: v1.34.10" L11: "cluster.go:125: [preflight] Running pre-flight checks" L12: "cluster.go:125: [WARNING SystemVerification]: cgroups v1 support is in maintenance mode, please migrate to cgroups v2" L13: "cluster.go:125: [WARNING Hostname]: hostname __ci-4081.3.9-n-cebca5c6f2__ could not be reached" L14: "cluster.go:125: [WARNING Hostname]: hostname __ci-4081.3.9-n-cebca5c6f2__: lookup ci-4081.3.9-n-cebca5c6f2 on 168.63.129.16:53: no such host" L15: "cluster.go:125: [WARNING Service-Kubelet]: kubelet service is not enabled, please run _systemctl enable kubelet.service_" L16: "cluster.go:125: [preflight] Pulling images required for setting up a Kubernetes cluster" L17: "cluster.go:125: [preflight] This might take a minute or two, depending on the speed of your internet connection" L18: "cluster.go:125: [preflight] You can also perform this action beforehand using _kubeadm config images pull_" L19: "cluster.go:125: W0724 23:41:29.339173 2731 checks.go:827] detected that the sandbox image __registry.k8s.io/pause:3.8__ of the container runtime is inconsistent with that used by kubeadm. It is rec?ommended to use __registry.k8s.io/pause:3.10.1__ as the CRI sandbox image." L20: "cluster.go:125: [certs] Using certificateDir folder __/etc/kubernetes/pki__" L21: "cluster.go:125: [certs] Generating __ca__ certificate and key" L22: "cluster.go:125: [certs] Generating __apiserver__ certificate and key" L23: "cluster.go:125: [certs] apiserver serving cert is signed for DNS names [ci-4081.3.9-n-cebca5c6f2 kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.?96.0.1 10.0.0.14]" L24: "cluster.go:125: [certs] Generating __apiserver-kubelet-client__ certificate and key" L25: "cluster.go:125: [certs] Generating __front-proxy-ca__ certificate and key" L26: "cluster.go:125: [certs] Generating __front-proxy-client__ certificate and key" L27: "cluster.go:125: [certs] External etcd mode: Skipping etcd/ca certificate authority generation" L28: "cluster.go:125: [certs] External etcd mode: Skipping etcd/server certificate generation" L29: "cluster.go:125: [certs] External etcd mode: Skipping etcd/peer certificate generation" L30: "cluster.go:125: [certs] External etcd mode: Skipping etcd/healthcheck-client certificate generation" L31: "cluster.go:125: [certs] External etcd mode: Skipping apiserver-etcd-client certificate generation" L32: "cluster.go:125: [certs] Generating __sa__ key and public key" L33: "cluster.go:125: [kubeconfig] Using kubeconfig folder __/etc/kubernetes__" L34: "cluster.go:125: [kubeconfig] Writing __admin.conf__ kubeconfig file" L35: "cluster.go:125: [kubeconfig] Writing __super-admin.conf__ kubeconfig file" L36: "cluster.go:125: [kubeconfig] Writing __kubelet.conf__ kubeconfig file" L37: "cluster.go:125: [kubeconfig] Writing __controller-manager.conf__ kubeconfig file" L38: "cluster.go:125: [kubeconfig] Writing __scheduler.conf__ kubeconfig file" L39: "cluster.go:125: [control-plane] Using manifest folder __/etc/kubernetes/manifests__" L40: "cluster.go:125: [control-plane] Creating static Pod manifest for __kube-apiserver__" L41: "cluster.go:125: [control-plane] Creating static Pod manifest for __kube-controller-manager__" L42: "cluster.go:125: [control-plane] Creating static Pod manifest for __kube-scheduler__" L43: "cluster.go:125: [kubelet-start] Writing kubelet environment file with flags to file __/var/lib/kubelet/kubeadm-flags.env__" L44: "cluster.go:125: [kubelet-start] Writing kubelet configuration to file __/var/lib/kubelet/instance-config.yaml__" L45: "cluster.go:125: [patches] Applied patch of type __application/strategic-merge-patch+json__ to target __kubeletconfiguration__" L46: "cluster.go:125: [kubelet-start] Writing kubelet configuration to file __/var/lib/kubelet/config.yaml__" L47: "cluster.go:125: [kubelet-start] Starting the kubelet" L48: "cluster.go:125: [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory __/etc/kubernetes/manifests__" L49: "cluster.go:125: [kubelet-check] Waiting for a healthy kubelet at http://127.0.0.1:10248/healthz. This can take up to 4m0s" L50: "cluster.go:125: [kubelet-check] The kubelet is healthy after 1.00094342s" L51: "cluster.go:125: [control-plane-check] Waiting for healthy control plane components. This can take up to 30m0s" L52: "cluster.go:125: [control-plane-check] Checking kube-apiserver at https://10.0.0.14:6443/livez" L53: "cluster.go:125: [control-plane-check] Checking kube-controller-manager at https://127.0.0.1:10257/healthz" L54: "cluster.go:125: [control-plane-check] Checking kube-scheduler at https://127.0.0.1:10259/livez" L55: "cluster.go:125: [control-plane-check] kube-controller-manager is healthy after 8.128531413s" L56: "cluster.go:125: [control-plane-check] kube-scheduler is healthy after 9.022787617s" L57: "cluster.go:125: [control-plane-check] kube-apiserver is healthy after 11.00156627s" L58: "cluster.go:125: [upload-config] Storing the configuration used in ConfigMap __kubeadm-config__ in the __kube-system__ Namespace" L59: "cluster.go:125: [kubelet] Creating a ConfigMap __kubelet-config__ in namespace kube-system with the configuration for the kubelets in the cluster" L60: "cluster.go:125: [upload-certs] Skipping phase. Please see --upload-certs" L61: "cluster.go:125: [mark-control-plane] Marking the node ci-4081.3.9-n-cebca5c6f2 as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-lo?ad-balancers]" L62: "cluster.go:125: [mark-control-plane] Marking the node ci-4081.3.9-n-cebca5c6f2 as control-plane by adding the taints [node-role.kubernetes.io/control-plane:NoSchedule]" L63: "cluster.go:125: [bootstrap-token] Using token: uh14ld.f0ksvaxxq5635h27" L64: "cluster.go:125: [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles" L65: "cluster.go:125: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes" L66: "cluster.go:125: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials" L67: "cluster.go:125: [bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token" L68: "cluster.go:125: [bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster" L69: "cluster.go:125: [bootstrap-token] Creating the __cluster-info__ ConfigMap in the __kube-public__ namespace" L70: "cluster.go:125: [kubelet-finalize] Updating __/etc/kubernetes/kubelet.conf__ to point to a rotatable kubelet client certificate and key" L71: "cluster.go:125: [addons] Applied essential addon: CoreDNS" L72: "cluster.go:125: [addons] Applied essential addon: kube-proxy" L73: "cluster.go:125: " L74: "cluster.go:125: Your Kubernetes control-plane has initialized successfully!" L75: "cluster.go:125: " L76: "cluster.go:125: To start using your cluster, you need to run the following as a regular user:" L77: "cluster.go:125: " L78: "cluster.go:125: mkdir -p $HOME/.kube" L79: "cluster.go:125: sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config" L80: "cluster.go:125: sudo chown $(id -u):$(id -g) $HOME/.kube/config" L81: "cluster.go:125: " L82: "cluster.go:125: Alternatively, if you are the root user, you can run:" L83: "cluster.go:125: " L84: "cluster.go:125: export KUBECONFIG=/etc/kubernetes/admin.conf" L85: "cluster.go:125: " L86: "cluster.go:125: You should now deploy a pod network to the cluster." L87: "cluster.go:125: Run __kubectl apply -f [podnetwork].yaml__ with one of the options listed at:" L88: "cluster.go:125: https://kubernetes.io/docs/concepts/cluster-administration/addons/" L89: "cluster.go:125: " L90: "cluster.go:125: Then you can join any number of worker nodes by running the following on each as root:" L91: "cluster.go:125: " L92: "cluster.go:125: kubeadm join 10.0.0.14:6443 --token uh14ld.f0ksvaxxq5635h27 _" L93: "cluster.go:125: --discovery-token-ca-cert-hash sha256:d4f989b37646698ee48c2d188e3b17ae0213aa02c106e9ae01a6103cae83d336 " L94: "cluster.go:125: namespace/tigera-operator created" L95: "cluster.go:125: serviceaccount/tigera-operator created" L96: "cluster.go:125: clusterrole.rbac.authorization.k8s.io/tigera-operator-secrets created" L97: "cluster.go:125: clusterrole.rbac.authorization.k8s.io/tigera-operator created" L98: "cluster.go:125: clusterrolebinding.rbac.authorization.k8s.io/tigera-operator created" L99: "cluster.go:125: rolebinding.rbac.authorization.k8s.io/tigera-operator-secrets created" L100: "cluster.go:125: deployment.apps/tigera-operator created" L101: "cluster.go:125: customresourcedefinition.apiextensions.k8s.io/installations.operator.tigera.io condition met" L102: "cluster.go:125: customresourcedefinition.apiextensions.k8s.io/installations.operator.tigera.io condition met" L103: "cluster.go:125: customresourcedefinition.apiextensions.k8s.io/apiservers.operator.tigera.io condition met" L104: "cluster.go:125: customresourcedefinition.apiextensions.k8s.io/apiservers.operator.tigera.io condition met" L105: "cluster.go:125: installation.operator.tigera.io/default created" L106: "cluster.go:125: apiserver.operator.tigera.io/default created" L107: "cluster.go:125: goldmane.operator.tigera.io/default created" L108: "cluster.go:125: whisker.operator.tigera.io/default created" L109: "cluster.go:125: W0724 23:44:02.272669 2479 joinconfiguration.go:112] [config] WARNING: Ignored configuration document with GroupVersionKind kubelet.config.k8s.io/v1beta1, Kind=KubeletConfiguration" L110: "cluster.go:125: [WARNING SystemVerification]: cgroups v1 support is in maintenance mode, please migrate to cgroups v2" L111: "cluster.go:125: [WARNING Service-Kubelet]: kubelet service is not enabled, please run _systemctl enable kubelet.service_" L112: "--- FAIL: kubeadm.v1.34.4.calico.cgroupv1.base/node_readiness (32.10s)" L113: "cluster.go:145: __kubectl get nodes | grep ___ Ready___| wc -l__ failed: output , status dial tcp 20.223.145.40:22: connect: connection refused" L114: "--- FAIL: kubeadm.v1.34.4.calico.cgroupv1.base/nginx_deployment (0.12s)" L115: "kubeadm.go:219: unable to deploy nginx: dial tcp 20.223.145.40:22: connect: connection refused" L116: "--- FAIL: kubeadm.v1.34.4.calico.cgroupv1.base/NFS_deployment (0.12s)" L117: "kubeadm.go:238: unable to add helm NFS repo: dial tcp 20.223.145.40:22: connect: connection refused_" L118: " " ```
🟢 ok **kubeadm.v1.34.4.cilium.base**; Succeeded: azure (2); Failed: azure (1)
                Diagnostic output for azure, run 1 ``` L1: " Error: _cluster.go:125: I0724 23:44:02.370422 2373 version.go:260] remote version is much newer: v1.36.3; falling back to: stable-1.34" L2: "cluster.go:125: [config/images] Pulled registry.k8s.io/kube-apiserver:v1.34.10" L3: "cluster.go:125: [config/images] Pulled registry.k8s.io/kube-controller-manager:v1.34.10" L4: "cluster.go:125: [config/images] Pulled registry.k8s.io/kube-scheduler:v1.34.10" L5: "cluster.go:125: [config/images] Pulled registry.k8s.io/kube-proxy:v1.34.10" L6: "cluster.go:125: [config/images] Pulled registry.k8s.io/coredns/coredns:v1.12.1" L7: "cluster.go:125: [config/images] Pulled registry.k8s.io/pause:3.10.1" L8: "kubeadm.go:197: unable to setup cluster: unable to run master script: wait: remote command exited without exit status or exit signal_" L9: " " ```
🟢 ok **kubeadm.v1.34.4.cilium.cgroupv1.base**; Succeeded: azure (1) 🟢 ok **kubeadm.v1.34.4.flannel.base**; Succeeded: azure (2); Failed: azure (1)
                Diagnostic output for azure, run 1 ``` L1: " Error: _kubeadm.go:197: unable to setup cluster: unable to create etcd node: PollUntilDone(ci-4081.3.9-n-cba7fca4a6): GET https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2?e/providers/Microsoft.Compute/locations/northeurope/operations/be599f2c-013b-48f4-837c-06f6afe6db6f" L2: "--------------------------------------------------------------------------------" L3: "RESPONSE 200: 200 OK" L4: "ERROR CODE: OperationPreempted" L5: "--------------------------------------------------------------------------------" L6: "{" L7: "__startTime__: __2026-07-24T23:43:18.2890734+00:00__," L8: "__endTime__: __2026-07-24T23:44:49.7580778+00:00__," L9: "__status__: __Canceled__," L10: "__error__: {" L11: "__code__: __OperationPreempted__," L12: "__message__: __Operation execution has been preempted by a more recent operation.__" L13: "}," L14: "__name__: __be599f2c-013b-48f4-837c-06f6afe6db6f__" L15: "}" L16: "--------------------------------------------------------------------------------_" L17: " " ```
🟢 ok **kubeadm.v1.34.4.flannel.cgroupv1.base**; Succeeded: azure (2); Failed: azure (1)
                Diagnostic output for azure, run 1 ``` L1: " Error: _cluster.go:125: I0724 23:40:54.029729 2621 version.go:260] remote version is much newer: v1.36.3; falling back to: stable-1.34" L2: "cluster.go:125: [config/images] Pulled registry.k8s.io/kube-apiserver:v1.34.10" L3: "cluster.go:125: [config/images] Pulled registry.k8s.io/kube-controller-manager:v1.34.10" L4: "cluster.go:125: [config/images] Pulled registry.k8s.io/kube-scheduler:v1.34.10" L5: "cluster.go:125: [config/images] Pulled registry.k8s.io/kube-proxy:v1.34.10" L6: "cluster.go:125: [config/images] Pulled registry.k8s.io/coredns/coredns:v1.12.1" L7: "cluster.go:125: [config/images] Pulled registry.k8s.io/pause:3.10.1" L8: "cluster.go:125: [config/images] Pulled registry.k8s.io/etcd:3.6.5-0" L9: "cluster.go:125: I0724 23:41:07.036272 2842 version.go:260] remote version is much newer: v1.36.3; falling back to: stable-1.34" L10: "cluster.go:125: [init] Using Kubernetes version: v1.34.10" L11: "cluster.go:125: [preflight] Running pre-flight checks" L12: "cluster.go:125: [WARNING SystemVerification]: cgroups v1 support is in maintenance mode, please migrate to cgroups v2" L13: "cluster.go:125: [WARNING Hostname]: hostname __ci-4081.3.9-n-301b0151a6__ could not be reached" L14: "cluster.go:125: [WARNING Hostname]: hostname __ci-4081.3.9-n-301b0151a6__: lookup ci-4081.3.9-n-301b0151a6 on 168.63.129.16:53: no such host" L15: "cluster.go:125: [WARNING Service-Kubelet]: kubelet service is not enabled, please run _systemctl enable kubelet.service_" L16: "cluster.go:125: [preflight] Pulling images required for setting up a Kubernetes cluster" L17: "cluster.go:125: [preflight] This might take a minute or two, depending on the speed of your internet connection" L18: "cluster.go:125: [preflight] You can also perform this action beforehand using _kubeadm config images pull_" L19: "cluster.go:125: W0724 23:41:07.169404 2842 checks.go:827] detected that the sandbox image __registry.k8s.io/pause:3.8__ of the container runtime is inconsistent with that used by kubeadm. It is rec?ommended to use __registry.k8s.io/pause:3.10.1__ as the CRI sandbox image." L20: "cluster.go:125: [certs] Using certificateDir folder __/etc/kubernetes/pki__" L21: "cluster.go:125: [certs] Generating __ca__ certificate and key" L22: "cluster.go:125: [certs] Generating __apiserver__ certificate and key" L23: "cluster.go:125: [certs] apiserver serving cert is signed for DNS names [ci-4081.3.9-n-301b0151a6 kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.?96.0.1 10.0.0.24]" L24: "cluster.go:125: [certs] Generating __apiserver-kubelet-client__ certificate and key" L25: "cluster.go:125: [certs] Generating __front-proxy-ca__ certificate and key" L26: "cluster.go:125: [certs] Generating __front-proxy-client__ certificate and key" L27: "cluster.go:125: [certs] External etcd mode: Skipping etcd/ca certificate authority generation" L28: "cluster.go:125: [certs] External etcd mode: Skipping etcd/server certificate generation" L29: "cluster.go:125: [certs] External etcd mode: Skipping etcd/peer certificate generation" L30: "cluster.go:125: [certs] External etcd mode: Skipping etcd/healthcheck-client certificate generation" L31: "cluster.go:125: [certs] External etcd mode: Skipping apiserver-etcd-client certificate generation" L32: "cluster.go:125: [certs] Generating __sa__ key and public key" L33: "cluster.go:125: [kubeconfig] Using kubeconfig folder __/etc/kubernetes__" L34: "cluster.go:125: [kubeconfig] Writing __admin.conf__ kubeconfig file" L35: "cluster.go:125: [kubeconfig] Writing __super-admin.conf__ kubeconfig file" L36: "cluster.go:125: [kubeconfig] Writing __kubelet.conf__ kubeconfig file" L37: "cluster.go:125: [kubeconfig] Writing __controller-manager.conf__ kubeconfig file" L38: "cluster.go:125: [kubeconfig] Writing __scheduler.conf__ kubeconfig file" L39: "cluster.go:125: [control-plane] Using manifest folder __/etc/kubernetes/manifests__" L40: "cluster.go:125: [control-plane] Creating static Pod manifest for __kube-apiserver__" L41: "cluster.go:125: [control-plane] Creating static Pod manifest for __kube-controller-manager__" L42: "cluster.go:125: [control-plane] Creating static Pod manifest for __kube-scheduler__" L43: "cluster.go:125: [kubelet-start] Writing kubelet environment file with flags to file __/var/lib/kubelet/kubeadm-flags.env__" L44: "cluster.go:125: [kubelet-start] Writing kubelet configuration to file __/var/lib/kubelet/instance-config.yaml__" L45: "cluster.go:125: [patches] Applied patch of type __application/strategic-merge-patch+json__ to target __kubeletconfiguration__" L46: "cluster.go:125: [kubelet-start] Writing kubelet configuration to file __/var/lib/kubelet/config.yaml__" L47: "cluster.go:125: [kubelet-start] Starting the kubelet" L48: "cluster.go:125: [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory __/etc/kubernetes/manifests__" L49: "cluster.go:125: [kubelet-check] Waiting for a healthy kubelet at http://127.0.0.1:10248/healthz. This can take up to 4m0s" L50: "cluster.go:125: [kubelet-check] The kubelet is healthy after 501.706877ms" L51: "cluster.go:125: [control-plane-check] Waiting for healthy control plane components. This can take up to 30m0s" L52: "cluster.go:125: [control-plane-check] Checking kube-apiserver at https://10.0.0.24:6443/livez" L53: "cluster.go:125: [control-plane-check] Checking kube-controller-manager at https://127.0.0.1:10257/healthz" L54: "cluster.go:125: [control-plane-check] Checking kube-scheduler at https://127.0.0.1:10259/livez" L55: "cluster.go:125: [control-plane-check] kube-controller-manager is healthy after 2.116206576s" L56: "cluster.go:125: [control-plane-check] kube-scheduler is healthy after 3.188748722s" L57: "cluster.go:125: [control-plane-check] kube-apiserver is healthy after 5.002069444s" L58: "cluster.go:125: [upload-config] Storing the configuration used in ConfigMap __kubeadm-config__ in the __kube-system__ Namespace" L59: "cluster.go:125: [kubelet] Creating a ConfigMap __kubelet-config__ in namespace kube-system with the configuration for the kubelets in the cluster" L60: "cluster.go:125: [upload-certs] Skipping phase. Please see --upload-certs" L61: "cluster.go:125: [mark-control-plane] Marking the node ci-4081.3.9-n-301b0151a6 as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-lo?ad-balancers]" L62: "cluster.go:125: [mark-control-plane] Marking the node ci-4081.3.9-n-301b0151a6 as control-plane by adding the taints [node-role.kubernetes.io/control-plane:NoSchedule]" L63: "cluster.go:125: [bootstrap-token] Using token: 2tcc4o.piuoj9kd53io7rhc" L64: "cluster.go:125: [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles" L65: "cluster.go:125: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes" L66: "cluster.go:125: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials" L67: "cluster.go:125: [bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token" L68: "cluster.go:125: [bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster" L69: "cluster.go:125: [bootstrap-token] Creating the __cluster-info__ ConfigMap in the __kube-public__ namespace" L70: "cluster.go:125: [kubelet-finalize] Updating __/etc/kubernetes/kubelet.conf__ to point to a rotatable kubelet client certificate and key" L71: "cluster.go:125: [addons] Applied essential addon: CoreDNS" L72: "cluster.go:125: [addons] Applied essential addon: kube-proxy" L73: "cluster.go:125: " L74: "cluster.go:125: Your Kubernetes control-plane has initialized successfully!" L75: "cluster.go:125: " L76: "cluster.go:125: To start using your cluster, you need to run the following as a regular user:" L77: "cluster.go:125: " L78: "cluster.go:125: mkdir -p $HOME/.kube" L79: "cluster.go:125: sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config" L80: "cluster.go:125: sudo chown $(id -u):$(id -g) $HOME/.kube/config" L81: "cluster.go:125: " L82: "cluster.go:125: Alternatively, if you are the root user, you can run:" L83: "cluster.go:125: " L84: "cluster.go:125: export KUBECONFIG=/etc/kubernetes/admin.conf" L85: "cluster.go:125: " L86: "cluster.go:125: You should now deploy a pod network to the cluster." L87: "cluster.go:125: Run __kubectl apply -f [podnetwork].yaml__ with one of the options listed at:" L88: "cluster.go:125: https://kubernetes.io/docs/concepts/cluster-administration/addons/" L89: "cluster.go:125: " L90: "cluster.go:125: Then you can join any number of worker nodes by running the following on each as root:" L91: "cluster.go:125: " L92: "cluster.go:125: kubeadm join 10.0.0.24:6443 --token 2tcc4o.piuoj9kd53io7rhc _" L93: "cluster.go:125: --discovery-token-ca-cert-hash sha256:0dea406fe3cc3c988dc71a9b152f00f0ce17cad23cc15d6a6cc7f4fb2650c9fa " L94: "cluster.go:125: namespace/kube-flannel created" L95: "cluster.go:125: clusterrole.rbac.authorization.k8s.io/flannel created" L96: "cluster.go:125: clusterrolebinding.rbac.authorization.k8s.io/flannel created" L97: "cluster.go:125: serviceaccount/flannel created" L98: "cluster.go:125: configmap/kube-flannel-cfg created" L99: "cluster.go:125: daemonset.apps/kube-flannel-ds created" L100: "kubeadm.go:197: unable to setup cluster: unable to create worker node: PUT https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kola-cluster-image-b4ba8baeca/?providers/Microsoft.Compute/virtualMachines/ci-4081.3.9-n-2a985059aa" L101: "--------------------------------------------------------------------------------" L102: "RESPONSE 409: 409 Conflict" L103: "ERROR CODE: OperationNotAllowed" L104: "--------------------------------------------------------------------------------" L105: "{" L106: "__error__: {" L107: "__code__: __OperationNotAllowed__," L108: "__message__: __Operation could not be completed as it results in exceeding approved standardDSv4Family Cores quota. Additional details - Deployment Model: Resource Manager, Location: northeurope, Curr?ent Limit: 65, Current Usage: 64, Additional Required: 2, (Minimum) New Limit Required: 66. Setup Alerts when Quota reaches threshold. Learn more at https://aka.ms/quotamonitoringalerting . Submit a r?equest for Quota increase at https://aka.ms/ProdportalCRP/#blade/Microsoft_Azure_Capacity/UsageAndQuota.ReactView/Parameters/%7B%22subscriptionId%22:%220e46bd28-a80f-4d3a-8200-d9eb8d80cb2e%22,%22comma?nd%22:%22openQuotaApprovalBlade%22,%22quotas%22:[%7B%22location%22:%22northeurope%22,%22providerId%22:%22Microsoft.Compute%22,%22resourceName%22:%22standardDSv4Family%22,%22quotaRequest%22:%7B%22prope?rties%22:%7B%22limit%22:66,%22unit%22:%22Count%22,%22name%22:%7B%22value%22:%22standardDSv4Family%22%7D%7D%7D%7D]%7D by specifying parameters listed in the ???Details??? section for deployment to succ?eed. Please read more about quota limits at https://docs.microsoft.com/en-us/azure/azure-supportability/per-vm-quota-requests__" L109: "}" L110: "}" L111: "--------------------------------------------------------------------------------_" L112: " " ```
🟢 ok **kubeadm.v1.35.1.calico.base**; Succeeded: azure (3); Failed: azure (1, 2)
                Diagnostic output for azure, run 2 ``` L1: " Error: _cluster.go:125: I0725 01:13:22.081165 2478 version.go:260] remote version is much newer: v1.36.3; falling back to: stable-1.35" L2: "cluster.go:125: [config/images] Pulled registry.k8s.io/kube-apiserver:v1.35.7" L3: "cluster.go:125: [config/images] Pulled registry.k8s.io/kube-controller-manager:v1.35.7" L4: "cluster.go:125: [config/images] Pulled registry.k8s.io/kube-scheduler:v1.35.7" L5: "cluster.go:125: [config/images] Pulled registry.k8s.io/kube-proxy:v1.35.7" L6: "cluster.go:125: [config/images] Pulled registry.k8s.io/coredns/coredns:v1.13.1" L7: "cluster.go:125: [config/images] Pulled registry.k8s.io/pause:3.10.1" L8: "cluster.go:125: [config/images] Pulled registry.k8s.io/etcd:3.6.6-0" L9: "cluster.go:125: I0725 01:13:34.243237 2689 version.go:260] remote version is much newer: v1.36.3; falling back to: stable-1.35" L10: "cluster.go:125: [init] Using Kubernetes version: v1.35.7" L11: "cluster.go:125: [preflight] Running pre-flight checks" L12: "cluster.go:125: [WARNING ContainerRuntimeVersion]: You must update your container runtime to a version that supports the CRI method RuntimeConfig. Falling back to using cgroupDriver from kubelet conf?ig will be removed in 1.36. For more information, see https://git.k8s.io/enhancements/keps/sig-node/4033-group-driver-detection-over-cri" L13: "cluster.go:125: [WARNING Hostname]: hostname __ci-4081.3.9-n-39febae43a__ could not be reached" L14: "cluster.go:125: [WARNING Hostname]: hostname __ci-4081.3.9-n-39febae43a__: lookup ci-4081.3.9-n-39febae43a on 168.63.129.16:53: no such host" L15: "cluster.go:125: [WARNING Service-kubelet]: kubelet service is not enabled, please run _systemctl enable kubelet.service_" L16: "cluster.go:125: [preflight] Pulling images required for setting up a Kubernetes cluster" L17: "cluster.go:125: [preflight] This might take a minute or two, depending on the speed of your internet connection" L18: "cluster.go:125: [preflight] You can also perform this action beforehand using _kubeadm config images pull_" L19: "cluster.go:125: W0725 01:13:34.589281 2689 checks.go:906] detected that the sandbox image __registry.k8s.io/pause:3.8__ of the container runtime is inconsistent with that used by kubeadm. It is rec?ommended to use __registry.k8s.io/pause:3.10.1__ as the CRI sandbox image." L20: "cluster.go:125: [certs] Using certificateDir folder __/etc/kubernetes/pki__" L21: "cluster.go:125: [certs] Generating __ca__ certificate and key" L22: "cluster.go:125: [certs] Generating __apiserver__ certificate and key" L23: "cluster.go:125: [certs] apiserver serving cert is signed for DNS names [ci-4081.3.9-n-39febae43a kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.?96.0.1 10.0.0.25]" L24: "cluster.go:125: [certs] Generating __apiserver-kubelet-client__ certificate and key" L25: "cluster.go:125: [certs] Generating __front-proxy-ca__ certificate and key" L26: "cluster.go:125: [certs] Generating __front-proxy-client__ certificate and key" L27: "cluster.go:125: [certs] External etcd mode: Skipping etcd/ca certificate authority generation" L28: "cluster.go:125: [certs] External etcd mode: Skipping etcd/server certificate generation" L29: "cluster.go:125: [certs] External etcd mode: Skipping etcd/peer certificate generation" L30: "cluster.go:125: [certs] External etcd mode: Skipping etcd/healthcheck-client certificate generation" L31: "cluster.go:125: [certs] External etcd mode: Skipping apiserver-etcd-client certificate generation" L32: "cluster.go:125: [certs] Generating __sa__ key and public key" L33: "cluster.go:125: [kubeconfig] Using kubeconfig folder __/etc/kubernetes__" L34: "cluster.go:125: [kubeconfig] Writing __admin.conf__ kubeconfig file" L35: "cluster.go:125: [kubeconfig] Writing __super-admin.conf__ kubeconfig file" L36: "cluster.go:125: [kubeconfig] Writing __kubelet.conf__ kubeconfig file" L37: "cluster.go:125: [kubeconfig] Writing __controller-manager.conf__ kubeconfig file" L38: "cluster.go:125: [kubeconfig] Writing __scheduler.conf__ kubeconfig file" L39: "cluster.go:125: [control-plane] Using manifest folder __/etc/kubernetes/manifests__" L40: "cluster.go:125: [control-plane] Creating static Pod manifest for __kube-apiserver__" L41: "cluster.go:125: [control-plane] Creating static Pod manifest for __kube-controller-manager__" L42: "cluster.go:125: [control-plane] Creating static Pod manifest for __kube-scheduler__" L43: "cluster.go:125: [kubelet-start] Writing kubelet environment file with flags to file __/var/lib/kubelet/kubeadm-flags.env__" L44: "cluster.go:125: [kubelet-start] Writing kubelet configuration to file __/var/lib/kubelet/instance-config.yaml__" L45: "cluster.go:125: [patches] Applied patch of type __application/strategic-merge-patch+json__ to target __kubeletconfiguration__" L46: "cluster.go:125: [kubelet-start] Writing kubelet configuration to file __/var/lib/kubelet/config.yaml__" L47: "cluster.go:125: [kubelet-start] Starting the kubelet" L48: "cluster.go:125: [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory __/etc/kubernetes/manifests__" L49: "cluster.go:125: [kubelet-check] Waiting for a healthy kubelet at http://127.0.0.1:10248/healthz. This can take up to 4m0s" L50: "cluster.go:125: [kubelet-check] The kubelet is healthy after 500.860986ms" L51: "cluster.go:125: [control-plane-check] Waiting for healthy control plane components. This can take up to 30m0s" L52: "cluster.go:125: [control-plane-check] Checking kube-apiserver at https://10.0.0.25:6443/livez" L53: "cluster.go:125: [control-plane-check] Checking kube-controller-manager at https://127.0.0.1:10257/healthz" L54: "cluster.go:125: [control-plane-check] Checking kube-scheduler at https://127.0.0.1:10259/livez" L55: "cluster.go:125: [control-plane-check] kube-controller-manager is healthy after 3.008602137s" L56: "cluster.go:125: [control-plane-check] kube-scheduler is healthy after 3.797184633s" L57: "cluster.go:125: [control-plane-check] kube-apiserver is healthy after 5.502053086s" L58: "cluster.go:125: [upload-config] Storing the configuration used in ConfigMap __kubeadm-config__ in the __kube-system__ Namespace" L59: "cluster.go:125: [kubelet] Creating a ConfigMap __kubelet-config__ in namespace kube-system with the configuration for the kubelets in the cluster" L60: "cluster.go:125: [upload-certs] Skipping phase. Please see --upload-certs" L61: "cluster.go:125: [mark-control-plane] Marking the node ci-4081.3.9-n-39febae43a as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-lo?ad-balancers]" L62: "cluster.go:125: [mark-control-plane] Marking the node ci-4081.3.9-n-39febae43a as control-plane by adding the taints [node-role.kubernetes.io/control-plane:NoSchedule]" L63: "cluster.go:125: [bootstrap-token] Using token: jemo2g.6om8yljyf5n6eg8p" L64: "cluster.go:125: [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles" L65: "cluster.go:125: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes" L66: "cluster.go:125: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials" L67: "cluster.go:125: [bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token" L68: "cluster.go:125: [bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster" L69: "cluster.go:125: [bootstrap-token] Creating the __cluster-info__ ConfigMap in the __kube-public__ namespace" L70: "cluster.go:125: [kubelet-finalize] Updating __/etc/kubernetes/kubelet.conf__ to point to a rotatable kubelet client certificate and key" L71: "cluster.go:125: [addons] Applied essential addon: CoreDNS" L72: "cluster.go:125: [addons] Applied essential addon: kube-proxy" L73: "cluster.go:125: " L74: "cluster.go:125: Your Kubernetes control-plane has initialized successfully!" L75: "cluster.go:125: " L76: "cluster.go:125: To start using your cluster, you need to run the following as a regular user:" L77: "cluster.go:125: " L78: "cluster.go:125: mkdir -p $HOME/.kube" L79: "cluster.go:125: sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config" L80: "cluster.go:125: sudo chown $(id -u):$(id -g) $HOME/.kube/config" L81: "cluster.go:125: " L82: "cluster.go:125: Alternatively, if you are the root user, you can run:" L83: "cluster.go:125: " L84: "cluster.go:125: export KUBECONFIG=/etc/kubernetes/admin.conf" L85: "cluster.go:125: " L86: "cluster.go:125: You should now deploy a pod network to the cluster." L87: "cluster.go:125: Run __kubectl apply -f [podnetwork].yaml__ with one of the options listed at:" L88: "cluster.go:125: https://kubernetes.io/docs/concepts/cluster-administration/addons/" L89: "cluster.go:125: " L90: "cluster.go:125: Then you can join any number of worker nodes by running the following on each as root:" L91: "cluster.go:125: " L92: "cluster.go:125: kubeadm join 10.0.0.25:6443 --token jemo2g.6om8yljyf5n6eg8p _" L93: "cluster.go:125: --discovery-token-ca-cert-hash sha256:b7cd6b46465908fa9ee987294b89aa6637601aed653023d0da147bec973575cb " L94: "cluster.go:125: namespace/tigera-operator created" L95: "cluster.go:125: serviceaccount/tigera-operator created" L96: "cluster.go:125: clusterrole.rbac.authorization.k8s.io/tigera-operator-secrets created" L97: "cluster.go:125: clusterrole.rbac.authorization.k8s.io/tigera-operator created" L98: "cluster.go:125: clusterrolebinding.rbac.authorization.k8s.io/tigera-operator created" L99: "cluster.go:125: rolebinding.rbac.authorization.k8s.io/tigera-operator-secrets created" L100: "cluster.go:125: deployment.apps/tigera-operator created" L101: "cluster.go:125: customresourcedefinition.apiextensions.k8s.io/installations.operator.tigera.io condition met" L102: "cluster.go:125: customresourcedefinition.apiextensions.k8s.io/installations.operator.tigera.io condition met" L103: "cluster.go:125: customresourcedefinition.apiextensions.k8s.io/apiservers.operator.tigera.io condition met" L104: "cluster.go:125: customresourcedefinition.apiextensions.k8s.io/apiservers.operator.tigera.io condition met" L105: "cluster.go:125: installation.operator.tigera.io/default created" L106: "cluster.go:125: apiserver.operator.tigera.io/default created" L107: "cluster.go:125: goldmane.operator.tigera.io/default created" L108: "cluster.go:125: whisker.operator.tigera.io/default created" L109: "kubeadm.go:197: unable to setup cluster: unable to create worker node: PUT https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kola-cluster-image-922124aacf/?providers/Microsoft.Compute/virtualMachines/ci-4081.3.9-n-2b519b4bed" L110: "--------------------------------------------------------------------------------" L111: "RESPONSE 409: 409 Conflict" L112: "ERROR CODE: OperationNotAllowed" L113: "--------------------------------------------------------------------------------" L114: "{" L115: "__error__: {" L116: "__code__: __OperationNotAllowed__," L117: "__message__: __Operation could not be completed as it results in exceeding approved standardDSv4Family Cores quota. Additional details - Deployment Model: Resource Manager, Location: northeurope, Curr?ent Limit: 65, Current Usage: 64, Additional Required: 2, (Minimum) New Limit Required: 66. Setup Alerts when Quota reaches threshold. Learn more at https://aka.ms/quotamonitoringalerting . Submit a r?equest for Quota increase at https://aka.ms/ProdportalCRP/#blade/Microsoft_Azure_Capacity/UsageAndQuota.ReactView/Parameters/%7B%22subscriptionId%22:%220e46bd28-a80f-4d3a-8200-d9eb8d80cb2e%22,%22comma?nd%22:%22openQuotaApprovalBlade%22,%22quotas%22:[%7B%22location%22:%22northeurope%22,%22providerId%22:%22Microsoft.Compute%22,%22resourceName%22:%22standardDSv4Family%22,%22quotaRequest%22:%7B%22prope?rties%22:%7B%22limit%22:66,%22unit%22:%22Count%22,%22name%22:%7B%22value%22:%22standardDSv4Family%22%7D%7D%7D%7D]%7D by specifying parameters listed in the ???Details??? section for deployment to succ?eed. Please read more about quota limits at https://docs.microsoft.com/en-us/azure/azure-supportability/per-vm-quota-requests__" L118: "}" L119: "}" L120: "--------------------------------------------------------------------------------_" L121: " " ```                 Diagnostic output for azure, run 1 ``` L1: " Error: _cluster.go:125: I0724 23:41:46.973431 2348 version.go:260] remote version is much newer: v1.36.3; falling back to: stable-1.35" L2: "cluster.go:125: [config/images] Pulled registry.k8s.io/kube-apiserver:v1.35.7" L3: "cluster.go:125: [config/images] Pulled registry.k8s.io/kube-controller-manager:v1.35.7" L4: "cluster.go:125: [config/images] Pulled registry.k8s.io/kube-scheduler:v1.35.7" L5: "cluster.go:125: [config/images] Pulled registry.k8s.io/kube-proxy:v1.35.7" L6: "cluster.go:125: [config/images] Pulled registry.k8s.io/coredns/coredns:v1.13.1" L7: "cluster.go:125: [config/images] Pulled registry.k8s.io/pause:3.10.1" L8: "cluster.go:125: [config/images] Pulled registry.k8s.io/etcd:3.6.6-0" L9: "cluster.go:125: I0724 23:42:21.253272 2655 version.go:260] remote version is much newer: v1.36.3; falling back to: stable-1.35" L10: "cluster.go:125: [init] Using Kubernetes version: v1.35.7" L11: "cluster.go:125: [preflight] Running pre-flight checks" L12: "cluster.go:125: [WARNING ContainerRuntimeVersion]: You must update your container runtime to a version that supports the CRI method RuntimeConfig. Falling back to using cgroupDriver from kubelet conf?ig will be removed in 1.36. For more information, see https://git.k8s.io/enhancements/keps/sig-node/4033-group-driver-detection-over-cri" L13: "cluster.go:125: [WARNING Hostname]: hostname __ci-4081.3.9-n-97f3835fae__ could not be reached" L14: "cluster.go:125: [WARNING Hostname]: hostname __ci-4081.3.9-n-97f3835fae__: lookup ci-4081.3.9-n-97f3835fae on 168.63.129.16:53: no such host" L15: "cluster.go:125: [WARNING Service-kubelet]: kubelet service is not enabled, please run _systemctl enable kubelet.service_" L16: "cluster.go:125: [preflight] Pulling images required for setting up a Kubernetes cluster" L17: "cluster.go:125: [preflight] This might take a minute or two, depending on the speed of your internet connection" L18: "cluster.go:125: [preflight] You can also perform this action beforehand using _kubeadm config images pull_" L19: "cluster.go:125: W0724 23:42:21.496201 2655 checks.go:906] detected that the sandbox image __registry.k8s.io/pause:3.8__ of the container runtime is inconsistent with that used by kubeadm. It is rec?ommended to use __registry.k8s.io/pause:3.10.1__ as the CRI sandbox image." L20: "cluster.go:125: [certs] Using certificateDir folder __/etc/kubernetes/pki__" L21: "cluster.go:125: [certs] Generating __ca__ certificate and key" L22: "cluster.go:125: [certs] Generating __apiserver__ certificate and key" L23: "cluster.go:125: [certs] apiserver serving cert is signed for DNS names [ci-4081.3.9-n-97f3835fae kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.?96.0.1 10.0.0.19]" L24: "cluster.go:125: [certs] Generating __apiserver-kubelet-client__ certificate and key" L25: "cluster.go:125: [certs] Generating __front-proxy-ca__ certificate and key" L26: "cluster.go:125: [certs] Generating __front-proxy-client__ certificate and key" L27: "cluster.go:125: [certs] External etcd mode: Skipping etcd/ca certificate authority generation" L28: "cluster.go:125: [certs] External etcd mode: Skipping etcd/server certificate generation" L29: "cluster.go:125: [certs] External etcd mode: Skipping etcd/peer certificate generation" L30: "cluster.go:125: [certs] External etcd mode: Skipping etcd/healthcheck-client certificate generation" L31: "cluster.go:125: [certs] External etcd mode: Skipping apiserver-etcd-client certificate generation" L32: "cluster.go:125: [certs] Generating __sa__ key and public key" L33: "cluster.go:125: [kubeconfig] Using kubeconfig folder __/etc/kubernetes__" L34: "cluster.go:125: [kubeconfig] Writing __admin.conf__ kubeconfig file" L35: "cluster.go:125: [kubeconfig] Writing __super-admin.conf__ kubeconfig file" L36: "cluster.go:125: [kubeconfig] Writing __kubelet.conf__ kubeconfig file" L37: "cluster.go:125: [kubeconfig] Writing __controller-manager.conf__ kubeconfig file" L38: "cluster.go:125: [kubeconfig] Writing __scheduler.conf__ kubeconfig file" L39: "cluster.go:125: [control-plane] Using manifest folder __/etc/kubernetes/manifests__" L40: "cluster.go:125: [control-plane] Creating static Pod manifest for __kube-apiserver__" L41: "cluster.go:125: [control-plane] Creating static Pod manifest for __kube-controller-manager__" L42: "cluster.go:125: [control-plane] Creating static Pod manifest for __kube-scheduler__" L43: "cluster.go:125: [kubelet-start] Writing kubelet environment file with flags to file __/var/lib/kubelet/kubeadm-flags.env__" L44: "cluster.go:125: [kubelet-start] Writing kubelet configuration to file __/var/lib/kubelet/instance-config.yaml__" L45: "cluster.go:125: [patches] Applied patch of type __application/strategic-merge-patch+json__ to target __kubeletconfiguration__" L46: "cluster.go:125: [kubelet-start] Writing kubelet configuration to file __/var/lib/kubelet/config.yaml__" L47: "cluster.go:125: [kubelet-start] Starting the kubelet" L48: "cluster.go:125: [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory __/etc/kubernetes/manifests__" L49: "cluster.go:125: [kubelet-check] Waiting for a healthy kubelet at http://127.0.0.1:10248/healthz. This can take up to 4m0s" L50: "cluster.go:125: [kubelet-check] The kubelet is healthy after 502.114941ms" L51: "cluster.go:125: [control-plane-check] Waiting for healthy control plane components. This can take up to 30m0s" L52: "cluster.go:125: [control-plane-check] Checking kube-apiserver at https://10.0.0.19:6443/livez" L53: "cluster.go:125: [control-plane-check] Checking kube-controller-manager at https://127.0.0.1:10257/healthz" L54: "cluster.go:125: [control-plane-check] Checking kube-scheduler at https://127.0.0.1:10259/livez" L55: "cluster.go:125: [control-plane-check] kube-scheduler is healthy after 5.516277264s" L56: "cluster.go:125: [control-plane-check] kube-controller-manager is healthy after 6.005835103s" L57: "cluster.go:125: [control-plane-check] kube-apiserver is healthy after 8.502208699s" L58: "cluster.go:125: [upload-config] Storing the configuration used in ConfigMap __kubeadm-config__ in the __kube-system__ Namespace" L59: "cluster.go:125: [kubelet] Creating a ConfigMap __kubelet-config__ in namespace kube-system with the configuration for the kubelets in the cluster" L60: "cluster.go:125: [upload-certs] Skipping phase. Please see --upload-certs" L61: "cluster.go:125: [mark-control-plane] Marking the node ci-4081.3.9-n-97f3835fae as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-lo?ad-balancers]" L62: "cluster.go:125: [mark-control-plane] Marking the node ci-4081.3.9-n-97f3835fae as control-plane by adding the taints [node-role.kubernetes.io/control-plane:NoSchedule]" L63: "cluster.go:125: [bootstrap-token] Using token: zciwt6.9zplmw7uokg4bu8e" L64: "cluster.go:125: [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles" L65: "cluster.go:125: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes" L66: "cluster.go:125: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials" L67: "cluster.go:125: [bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token" L68: "cluster.go:125: [bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster" L69: "cluster.go:125: [bootstrap-token] Creating the __cluster-info__ ConfigMap in the __kube-public__ namespace" L70: "cluster.go:125: [kubelet-finalize] Updating __/etc/kubernetes/kubelet.conf__ to point to a rotatable kubelet client certificate and key" L71: "cluster.go:125: [addons] Applied essential addon: CoreDNS" L72: "cluster.go:125: [addons] Applied essential addon: kube-proxy" L73: "cluster.go:125: " L74: "cluster.go:125: Your Kubernetes control-plane has initialized successfully!" L75: "cluster.go:125: " L76: "cluster.go:125: To start using your cluster, you need to run the following as a regular user:" L77: "cluster.go:125: " L78: "cluster.go:125: mkdir -p $HOME/.kube" L79: "cluster.go:125: sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config" L80: "cluster.go:125: sudo chown $(id -u):$(id -g) $HOME/.kube/config" L81: "cluster.go:125: " L82: "cluster.go:125: Alternatively, if you are the root user, you can run:" L83: "cluster.go:125: " L84: "cluster.go:125: export KUBECONFIG=/etc/kubernetes/admin.conf" L85: "cluster.go:125: " L86: "cluster.go:125: You should now deploy a pod network to the cluster." L87: "cluster.go:125: Run __kubectl apply -f [podnetwork].yaml__ with one of the options listed at:" L88: "cluster.go:125: https://kubernetes.io/docs/concepts/cluster-administration/addons/" L89: "cluster.go:125: " L90: "cluster.go:125: Then you can join any number of worker nodes by running the following on each as root:" L91: "cluster.go:125: " L92: "cluster.go:125: kubeadm join 10.0.0.19:6443 --token zciwt6.9zplmw7uokg4bu8e _" L93: "cluster.go:125: --discovery-token-ca-cert-hash sha256:4e625982976fcdc9d4e95771354cce618b90a581d3fc9f5845e875f66e400afe " L94: "cluster.go:125: namespace/tigera-operator created" L95: "cluster.go:125: serviceaccount/tigera-operator created" L96: "cluster.go:125: clusterrole.rbac.authorization.k8s.io/tigera-operator-secrets created" L97: "cluster.go:125: clusterrole.rbac.authorization.k8s.io/tigera-operator created" L98: "cluster.go:125: clusterrolebinding.rbac.authorization.k8s.io/tigera-operator created" L99: "cluster.go:125: rolebinding.rbac.authorization.k8s.io/tigera-operator-secrets created" L100: "cluster.go:125: deployment.apps/tigera-operator created" L101: "cluster.go:125: customresourcedefinition.apiextensions.k8s.io/installations.operator.tigera.io condition met" L102: "cluster.go:125: customresourcedefinition.apiextensions.k8s.io/installations.operator.tigera.io condition met" L103: "cluster.go:125: customresourcedefinition.apiextensions.k8s.io/apiservers.operator.tigera.io condition met" L104: "cluster.go:125: customresourcedefinition.apiextensions.k8s.io/apiservers.operator.tigera.io condition met" L105: "cluster.go:125: installation.operator.tigera.io/default created" L106: "cluster.go:125: apiserver.operator.tigera.io/default created" L107: "cluster.go:125: goldmane.operator.tigera.io/default created" L108: "cluster.go:125: whisker.operator.tigera.io/default created" L109: "kubeadm.go:197: unable to setup cluster: unable to create worker node: waiting for machine to become active: GET https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourc?eGroups/kola-cluster-image-b4ba8baeca/providers/Microsoft.Compute/virtualMachines/ci-4081.3.9-n-7c505bdaa7" L110: "--------------------------------------------------------------------------------" L111: "RESPONSE 404: 404 Not Found" L112: "ERROR CODE: NotFound" L113: "--------------------------------------------------------------------------------" L114: "{" L115: "__error__: {" L116: "__code__: __NotFound__," L117: "__message__: __The entity was not found in this Azure location.__" L118: "}" L119: "}" L120: "--------------------------------------------------------------------------------_" L121: " " ```
🟢 ok **kubeadm.v1.35.1.cilium.base**; Succeeded: azure (2); Failed: azure (1)
                Diagnostic output for azure, run 1 ``` L1: " Error: _cluster.go:125: I0724 23:40:25.297580 2375 version.go:260] remote version is much newer: v1.36.3; falling back to: stable-1.35" L2: "cluster.go:125: [config/images] Pulled registry.k8s.io/kube-apiserver:v1.35.7" L3: "cluster.go:125: [config/images] Pulled registry.k8s.io/kube-controller-manager:v1.35.7" L4: "cluster.go:125: [config/images] Pulled registry.k8s.io/kube-scheduler:v1.35.7" L5: "cluster.go:125: [config/images] Pulled registry.k8s.io/kube-proxy:v1.35.7" L6: "cluster.go:125: [config/images] Pulled registry.k8s.io/coredns/coredns:v1.13.1" L7: "cluster.go:125: [config/images] Pulled registry.k8s.io/pause:3.10.1" L8: "cluster.go:125: [config/images] Pulled registry.k8s.io/etcd:3.6.6-0" L9: "cluster.go:125: I0724 23:40:53.728963 2709 version.go:260] remote version is much newer: v1.36.3; falling back to: stable-1.35" L10: "cluster.go:125: [init] Using Kubernetes version: v1.35.7" L11: "cluster.go:125: [preflight] Running pre-flight checks" L12: "cluster.go:125: [WARNING ContainerRuntimeVersion]: You must update your container runtime to a version that supports the CRI method RuntimeConfig. Falling back to using cgroupDriver from kubelet conf?ig will be removed in 1.36. For more information, see https://git.k8s.io/enhancements/keps/sig-node/4033-group-driver-detection-over-cri" L13: "cluster.go:125: [WARNING Hostname]: hostname __ci-4081.3.9-n-2134e5c62b__ could not be reached" L14: "cluster.go:125: [WARNING Hostname]: hostname __ci-4081.3.9-n-2134e5c62b__: lookup ci-4081.3.9-n-2134e5c62b on 168.63.129.16:53: no such host" L15: "cluster.go:125: [WARNING Service-kubelet]: kubelet service is not enabled, please run _systemctl enable kubelet.service_" L16: "cluster.go:125: [preflight] Pulling images required for setting up a Kubernetes cluster" L17: "cluster.go:125: [preflight] This might take a minute or two, depending on the speed of your internet connection" L18: "cluster.go:125: [preflight] You can also perform this action beforehand using _kubeadm config images pull_" L19: "cluster.go:125: W0724 23:40:53.984787 2709 checks.go:906] detected that the sandbox image __registry.k8s.io/pause:3.8__ of the container runtime is inconsistent with that used by kubeadm. It is rec?ommended to use __registry.k8s.io/pause:3.10.1__ as the CRI sandbox image." L20: "cluster.go:125: [certs] Using certificateDir folder __/etc/kubernetes/pki__" L21: "cluster.go:125: [certs] Generating __ca__ certificate and key" L22: "cluster.go:125: [certs] Generating __apiserver__ certificate and key" L23: "cluster.go:125: [certs] apiserver serving cert is signed for DNS names [ci-4081.3.9-n-2134e5c62b kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.?96.0.1 10.0.0.25]" L24: "cluster.go:125: [certs] Generating __apiserver-kubelet-client__ certificate and key" L25: "cluster.go:125: [certs] Generating __front-proxy-ca__ certificate and key" L26: "cluster.go:125: [certs] Generating __front-proxy-client__ certificate and key" L27: "cluster.go:125: [certs] External etcd mode: Skipping etcd/ca certificate authority generation" L28: "cluster.go:125: [certs] External etcd mode: Skipping etcd/server certificate generation" L29: "cluster.go:125: [certs] External etcd mode: Skipping etcd/peer certificate generation" L30: "cluster.go:125: [certs] External etcd mode: Skipping etcd/healthcheck-client certificate generation" L31: "cluster.go:125: [certs] External etcd mode: Skipping apiserver-etcd-client certificate generation" L32: "cluster.go:125: [certs] Generating __sa__ key and public key" L33: "cluster.go:125: [kubeconfig] Using kubeconfig folder __/etc/kubernetes__" L34: "cluster.go:125: [kubeconfig] Writing __admin.conf__ kubeconfig file" L35: "cluster.go:125: [kubeconfig] Writing __super-admin.conf__ kubeconfig file" L36: "cluster.go:125: [kubeconfig] Writing __kubelet.conf__ kubeconfig file" L37: "cluster.go:125: [kubeconfig] Writing __controller-manager.conf__ kubeconfig file" L38: "cluster.go:125: [kubeconfig] Writing __scheduler.conf__ kubeconfig file" L39: "cluster.go:125: [control-plane] Using manifest folder __/etc/kubernetes/manifests__" L40: "cluster.go:125: [control-plane] Creating static Pod manifest for __kube-apiserver__" L41: "cluster.go:125: [control-plane] Creating static Pod manifest for __kube-controller-manager__" L42: "cluster.go:125: [control-plane] Creating static Pod manifest for __kube-scheduler__" L43: "cluster.go:125: [kubelet-start] Writing kubelet environment file with flags to file __/var/lib/kubelet/kubeadm-flags.env__" L44: "cluster.go:125: [kubelet-start] Writing kubelet configuration to file __/var/lib/kubelet/instance-config.yaml__" L45: "cluster.go:125: [patches] Applied patch of type __application/strategic-merge-patch+json__ to target __kubeletconfiguration__" L46: "cluster.go:125: [kubelet-start] Writing kubelet configuration to file __/var/lib/kubelet/config.yaml__" L47: "cluster.go:125: [kubelet-start] Starting the kubelet" L48: "cluster.go:125: [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory __/etc/kubernetes/manifests__" L49: "cluster.go:125: [kubelet-check] Waiting for a healthy kubelet at http://127.0.0.1:10248/healthz. This can take up to 4m0s" L50: "cluster.go:125: [kubelet-check] The kubelet is healthy after 501.523483ms" L51: "cluster.go:125: [control-plane-check] Waiting for healthy control plane components. This can take up to 30m0s" L52: "cluster.go:125: [control-plane-check] Checking kube-apiserver at https://10.0.0.25:6443/livez" L53: "cluster.go:125: [control-plane-check] Checking kube-controller-manager at https://127.0.0.1:10257/healthz" L54: "cluster.go:125: [control-plane-check] Checking kube-scheduler at https://127.0.0.1:10259/livez" L55: "cluster.go:125: [control-plane-check] kube-controller-manager is healthy after 2.512153975s" L56: "cluster.go:125: [control-plane-check] kube-scheduler is healthy after 3.223535014s" L57: "cluster.go:125: [control-plane-check] kube-apiserver is healthy after 5.001680954s" L58: "cluster.go:125: [upload-config] Storing the configuration used in ConfigMap __kubeadm-config__ in the __kube-system__ Namespace" L59: "cluster.go:125: [kubelet] Creating a ConfigMap __kubelet-config__ in namespace kube-system with the configuration for the kubelets in the cluster" L60: "cluster.go:125: [upload-certs] Skipping phase. Please see --upload-certs" L61: "cluster.go:125: [mark-control-plane] Marking the node ci-4081.3.9-n-2134e5c62b as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-lo?ad-balancers]" L62: "cluster.go:125: [mark-control-plane] Marking the node ci-4081.3.9-n-2134e5c62b as control-plane by adding the taints [node-role.kubernetes.io/control-plane:NoSchedule]" L63: "cluster.go:125: [bootstrap-token] Using token: apqyim.lfo95h09xhl1oq2l" L64: "cluster.go:125: [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles" L65: "cluster.go:125: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes" L66: "cluster.go:125: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials" L67: "cluster.go:125: [bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token" L68: "cluster.go:125: [bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster" L69: "cluster.go:125: [bootstrap-token] Creating the __cluster-info__ ConfigMap in the __kube-public__ namespace" L70: "cluster.go:125: [kubelet-finalize] Updating __/etc/kubernetes/kubelet.conf__ to point to a rotatable kubelet client certificate and key" L71: "cluster.go:125: [addons] Applied essential addon: CoreDNS" L72: "cluster.go:125: [addons] Applied essential addon: kube-proxy" L73: "cluster.go:125: " L74: "cluster.go:125: Your Kubernetes control-plane has initialized successfully!" L75: "cluster.go:125: " L76: "cluster.go:125: To start using your cluster, you need to run the following as a regular user:" L77: "cluster.go:125: " L78: "cluster.go:125: mkdir -p $HOME/.kube" L79: "cluster.go:125: sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config" L80: "cluster.go:125: sudo chown $(id -u):$(id -g) $HOME/.kube/config" L81: "cluster.go:125: " L82: "cluster.go:125: Alternatively, if you are the root user, you can run:" L83: "cluster.go:125: " L84: "cluster.go:125: export KUBECONFIG=/etc/kubernetes/admin.conf" L85: "cluster.go:125: " L86: "cluster.go:125: You should now deploy a pod network to the cluster." L87: "cluster.go:125: Run __kubectl apply -f [podnetwork].yaml__ with one of the options listed at:" L88: "cluster.go:125: https://kubernetes.io/docs/concepts/cluster-administration/addons/" L89: "cluster.go:125: " L90: "cluster.go:125: Then you can join any number of worker nodes by running the following on each as root:" L91: "cluster.go:125: " L92: "cluster.go:125: kubeadm join 10.0.0.25:6443 --token apqyim.lfo95h09xhl1oq2l _" L93: "cluster.go:125: --discovery-token-ca-cert-hash sha256:c39636493f736072f734a45c411c8499f95968c75633b82e54420d2a40ee116c " L94: "cluster.go:125: i Using Cilium version 1.12.5" L95: "cluster.go:125: ? Auto-detected cluster name: kubernetes" L96: "cluster.go:125: ? Auto-detected datapath mode: tunnel" L97: "cluster.go:125: ? Auto-detected kube-proxy has been installed" L98: "cluster.go:125: i helm template --namespace kube-system cilium cilium/cilium --version 1.12.5 --set cluster.id=0,cluster.name=kubernetes,encryption.nodeEncryption=false,extraConfig.cluster-pool-ipv4-?cidr=192.168.0.0/17,extraConfig.enable-endpoint-routes=true,kubeProxyReplacement=disabled,operator.replicas=1,serviceAccounts.cilium.name=cilium,serviceAccounts.operator.name=cilium-operator,tunnel=vx?lan" L99: "cluster.go:125: i Storing helm values file in kube-system/cilium-cli-helm-values Secret" L100: "cluster.go:125: ? Created CA in secret cilium-ca" L101: "cluster.go:125: ? Generating certificates for Hubble..." L102: "cluster.go:125: ? Creating Service accounts..." L103: "cluster.go:125: ? Creating Cluster roles..." L104: "cluster.go:125: ? Creating ConfigMap for Cilium version 1.12.5..." L105: "cluster.go:125: i Manual overwrite in ConfigMap: enable-endpoint-routes=true" L106: "cluster.go:125: i Manual overwrite in ConfigMap: cluster-pool-ipv4-cidr=192.168.0.0/17" L107: "cluster.go:125: ? Creating Agent DaemonSet..." L108: "cluster.go:125: level=warning msg=__spec.template.metadata.annotations[container.apparmor.security.beta.kubernetes.io/mount-cgroup]: deprecated since v1.30; use the ___appArmorProfile___ field instead?__ subsys=klog" L109: "cluster.go:125: level=warning msg=__spec.template.metadata.annotations[container.apparmor.security.beta.kubernetes.io/apply-sysctl-overwrites]: deprecated since v1.30; use the ___appArmorProfile___ fi?eld instead__ subsys=klog" L110: "cluster.go:125: level=warning msg=__spec.template.metadata.annotations[container.apparmor.security.beta.kubernetes.io/clean-cilium-state]: deprecated since v1.30; use the ___appArmorProfile___ field i?nstead__ subsys=klog" L111: "cluster.go:125: level=warning msg=__spec.template.metadata.annotations[container.apparmor.security.beta.kubernetes.io/cilium-agent]: deprecated since v1.30; use the ___appArmorProfile___ field instead?__ subsys=klog" L112: "cluster.go:125: ? Creating Operator Deployment..." L113: "cluster.go:125: ? Waiting for Cilium to be installed and ready..." L114: "cluster.go:125: ? Cilium was successfully installed! Run _cilium status_ to view installation health" L115: "cluster.go:125: ?[33m /??_" L116: "cluster.go:125: ?[36m /???[33m___/?[32m??_?[0m Cilium: ?[32mOK?[0m" L117: "cluster.go:125: ?[36m ___?[31m/??_?[32m__/?[0m Operator: ?[32mOK?[0m" L118: "cluster.go:125: ?[32m /???[31m___/?[35m??_?[0m Hubble: ?[36mdisabled?[0m" L119: "cluster.go:125: ?[32m ___?[34m/??_?[35m__/?[0m ClusterMesh: ?[36mdisabled?[0m" L120: "cluster.go:125: ?[34m ___/" L121: "cluster.go:125: ?[0m" L122: "cluster.go:125: Deployment cilium-operator " L123: "cluster.go:125: DaemonSet cilium " L124: "cluster.go:125: Containers: cilium " L125: "cluster.go:125: cilium-operator " L126: "cluster.go:125: Cluster Pods: 0/0 managed by Cilium" L127: "kubeadm.go:197: unable to setup cluster: unable to create worker node: PUT https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kola-cluster-image-b4ba8baeca/?providers/Microsoft.Compute/virtualMachines/ci-4081.3.9-n-1d192fa86c" L128: "--------------------------------------------------------------------------------" L129: "RESPONSE 409: 409 Conflict" L130: "ERROR CODE: OperationNotAllowed" L131: "--------------------------------------------------------------------------------" L132: "{" L133: "__error__: {" L134: "__code__: __OperationNotAllowed__," L135: "__message__: __Operation could not be completed as it results in exceeding approved standardDSv4Family Cores quota. Additional details - Deployment Model: Resource Manager, Location: northeurope, Curr?ent Limit: 65, Current Usage: 64, Additional Required: 2, (Minimum) New Limit Required: 66. Setup Alerts when Quota reaches threshold. Learn more at https://aka.ms/quotamonitoringalerting . Submit a r?equest for Quota increase at https://aka.ms/ProdportalCRP/#blade/Microsoft_Azure_Capacity/UsageAndQuota.ReactView/Parameters/%7B%22subscriptionId%22:%220e46bd28-a80f-4d3a-8200-d9eb8d80cb2e%22,%22comma?nd%22:%22openQuotaApprovalBlade%22,%22quotas%22:[%7B%22location%22:%22northeurope%22,%22providerId%22:%22Microsoft.Compute%22,%22resourceName%22:%22standardDSv4Family%22,%22quotaRequest%22:%7B%22prope?rties%22:%7B%22limit%22:66,%22unit%22:%22Count%22,%22name%22:%7B%22value%22:%22standardDSv4Family%22%7D%7D%7D%7D]%7D by specifying parameters listed in the ???Details??? section for deployment to succ?eed. Please read more about quota limits at https://docs.microsoft.com/en-us/azure/azure-supportability/per-vm-quota-requests__" L136: "}" L137: "}" L138: "--------------------------------------------------------------------------------_" L139: " " ```
🟢 ok **kubeadm.v1.35.1.flannel.base**; Succeeded: azure (3); Failed: azure (1, 2)
                Diagnostic output for azure, run 2 ``` L1: " Error: _kubeadm.go:197: unable to setup cluster: unable to create master node: PUT https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kola-cluster-image-922?124aacf/providers/Microsoft.Compute/virtualMachines/ci-4081.3.9-n-fb136c7e3b" L2: "--------------------------------------------------------------------------------" L3: "RESPONSE 409: 409 Conflict" L4: "ERROR CODE: OperationNotAllowed" L5: "--------------------------------------------------------------------------------" L6: "{" L7: "__error__: {" L8: "__code__: __OperationNotAllowed__," L9: "__message__: __Operation could not be completed as it results in exceeding approved standardDSv4Family Cores quota. Additional details - Deployment Model: Resource Manager, Location: northeurope, Curr?ent Limit: 65, Current Usage: 64, Additional Required: 2, (Minimum) New Limit Required: 66. Setup Alerts when Quota reaches threshold. Learn more at https://aka.ms/quotamonitoringalerting . Submit a r?equest for Quota increase at https://aka.ms/ProdportalCRP/#blade/Microsoft_Azure_Capacity/UsageAndQuota.ReactView/Parameters/%7B%22subscriptionId%22:%220e46bd28-a80f-4d3a-8200-d9eb8d80cb2e%22,%22comma?nd%22:%22openQuotaApprovalBlade%22,%22quotas%22:[%7B%22location%22:%22northeurope%22,%22providerId%22:%22Microsoft.Compute%22,%22resourceName%22:%22standardDSv4Family%22,%22quotaRequest%22:%7B%22prope?rties%22:%7B%22limit%22:66,%22unit%22:%22Count%22,%22name%22:%7B%22value%22:%22standardDSv4Family%22%7D%7D%7D%7D]%7D by specifying parameters listed in the ???Details??? section for deployment to succ?eed. Please read more about quota limits at https://docs.microsoft.com/en-us/azure/azure-supportability/per-vm-quota-requests__" L10: "}" L11: "}" L12: "--------------------------------------------------------------------------------_" L13: " " ```                 Diagnostic output for azure, run 1 ``` L1: " Error: _kubeadm.go:197: unable to setup cluster: unable to create etcd node: PUT https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kola-cluster-image-b4ba8?baeca/providers/Microsoft.Compute/virtualMachines/ci-4081.3.9-n-0723dd90b9" L2: "--------------------------------------------------------------------------------" L3: "RESPONSE 409: 409 Conflict" L4: "ERROR CODE: OperationNotAllowed" L5: "--------------------------------------------------------------------------------" L6: "{" L7: "__error__: {" L8: "__code__: __OperationNotAllowed__," L9: "__message__: __Operation could not be completed as it results in exceeding approved standardDSv4Family Cores quota. Additional details - Deployment Model: Resource Manager, Location: northeurope, Curr?ent Limit: 65, Current Usage: 64, Additional Required: 2, (Minimum) New Limit Required: 66. Setup Alerts when Quota reaches threshold. Learn more at https://aka.ms/quotamonitoringalerting . Submit a r?equest for Quota increase at https://aka.ms/ProdportalCRP/#blade/Microsoft_Azure_Capacity/UsageAndQuota.ReactView/Parameters/%7B%22subscriptionId%22:%220e46bd28-a80f-4d3a-8200-d9eb8d80cb2e%22,%22comma?nd%22:%22openQuotaApprovalBlade%22,%22quotas%22:[%7B%22location%22:%22northeurope%22,%22providerId%22:%22Microsoft.Compute%22,%22resourceName%22:%22standardDSv4Family%22,%22quotaRequest%22:%7B%22prope?rties%22:%7B%22limit%22:66,%22unit%22:%22Count%22,%22name%22:%7B%22value%22:%22standardDSv4Family%22%7D%7D%7D%7D]%7D by specifying parameters listed in the ???Details??? section for deployment to succ?eed. Please read more about quota limits at https://docs.microsoft.com/en-us/azure/azure-supportability/per-vm-quota-requests__" L10: "}" L11: "}" L12: "--------------------------------------------------------------------------------_" L13: " " ```
🟢 ok **kubeadm.v1.36.1.calico.base**; Succeeded: azure (3); Failed: azure (1, 2)
                Diagnostic output for azure, run 2 ``` L1: " Error: _cluster.go:125: [config/images] Pulled registry.k8s.io/kube-apiserver:v1.36.3" L2: "cluster.go:125: [config/images] Pulled registry.k8s.io/kube-controller-manager:v1.36.3" L3: "cluster.go:125: [config/images] Pulled registry.k8s.io/kube-scheduler:v1.36.3" L4: "cluster.go:125: [config/images] Pulled registry.k8s.io/kube-proxy:v1.36.3" L5: "cluster.go:125: [config/images] Pulled registry.k8s.io/coredns/coredns:v1.14.2" L6: "cluster.go:125: [config/images] Pulled registry.k8s.io/pause:3.10.2" L7: "cluster.go:125: [config/images] Pulled registry.k8s.io/etcd:3.6.8-0" L8: "cluster.go:125: [init] Using Kubernetes version: v1.36.3" L9: "cluster.go:125: [preflight] Running pre-flight checks" L10: "cluster.go:125: E0725 01:13:03.816212 2619 remote_runtime.go:1177] __RuntimeConfig from runtime service failed__ err=__rpc error: code = Unimplemented desc = unknown method RuntimeConfig for servic?e runtime.v1.RuntimeService__" L11: "cluster.go:125: [WARNING ContainerRuntimeVersion]: You must update your container runtime to a version that supports the CRI method RuntimeConfig. Falling back to using cgroupDriver from kubelet conf?ig will be removed in 1.37. For more information, see https://git.k8s.io/enhancements/keps/sig-node/4033-group-driver-detection-over-cri" L12: "cluster.go:125: [WARNING Hostname]: hostname __ci-4081.3.9-n-2df587b7ea__ could not be reached" L13: "cluster.go:125: [WARNING Hostname]: hostname __ci-4081.3.9-n-2df587b7ea__: lookup ci-4081.3.9-n-2df587b7ea on 168.63.129.16:53: no such host" L14: "cluster.go:125: [WARNING Service-kubelet]: kubelet service is not enabled, please run _systemctl enable kubelet.service_" L15: "cluster.go:125: [preflight] Pulling images required for setting up a Kubernetes cluster" L16: "cluster.go:125: [preflight] This might take a minute or two, depending on the speed of your internet connection" L17: "cluster.go:125: [preflight] You can also perform this action beforehand using _kubeadm config images pull_" L18: "cluster.go:125: W0725 01:13:03.964559 2619 checks.go:907] detected that the sandbox image __registry.k8s.io/pause:3.8__ of the container runtime is inconsistent with that used by kubeadm. It is rec?ommended to use __registry.k8s.io/pause:3.10.2__ as the CRI sandbox image." L19: "cluster.go:125: [certs] Using certificateDir folder __/etc/kubernetes/pki__" L20: "cluster.go:125: [certs] Generating __ca__ certificate and key" L21: "cluster.go:125: [certs] Generating __apiserver__ certificate and key" L22: "cluster.go:125: [certs] apiserver serving cert is signed for DNS names [ci-4081.3.9-n-2df587b7ea kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.?96.0.1 10.0.0.27]" L23: "cluster.go:125: [certs] Generating __apiserver-kubelet-client__ certificate and key" L24: "cluster.go:125: [certs] Generating __front-proxy-ca__ certificate and key" L25: "cluster.go:125: [certs] Generating __front-proxy-client__ certificate and key" L26: "cluster.go:125: [certs] External etcd mode: Skipping etcd/ca certificate authority generation" L27: "cluster.go:125: [certs] External etcd mode: Skipping etcd/server certificate generation" L28: "cluster.go:125: [certs] External etcd mode: Skipping etcd/peer certificate generation" L29: "cluster.go:125: [certs] External etcd mode: Skipping etcd/healthcheck-client certificate generation" L30: "cluster.go:125: [certs] External etcd mode: Skipping apiserver-etcd-client certificate generation" L31: "cluster.go:125: [certs] Generating __sa__ key and public key" L32: "cluster.go:125: [kubeconfig] Using kubeconfig folder __/etc/kubernetes__" L33: "cluster.go:125: [kubeconfig] Writing __admin.conf__ kubeconfig file" L34: "cluster.go:125: [kubeconfig] Writing __super-admin.conf__ kubeconfig file" L35: "cluster.go:125: [kubeconfig] Writing __kubelet.conf__ kubeconfig file" L36: "cluster.go:125: [kubeconfig] Writing __controller-manager.conf__ kubeconfig file" L37: "cluster.go:125: [kubeconfig] Writing __scheduler.conf__ kubeconfig file" L38: "cluster.go:125: [control-plane] Using manifest folder __/etc/kubernetes/manifests__" L39: "cluster.go:125: [control-plane] Creating static Pod manifest for __kube-apiserver__" L40: "cluster.go:125: [control-plane] Creating static Pod manifest for __kube-controller-manager__" L41: "cluster.go:125: [control-plane] Creating static Pod manifest for __kube-scheduler__" L42: "cluster.go:125: [kubelet-start] Writing kubelet environment file with flags to file __/var/lib/kubelet/kubeadm-flags.env__" L43: "cluster.go:125: [kubelet-start] Writing kubelet configuration to file __/var/lib/kubelet/instance-config.yaml__" L44: "cluster.go:125: [patches] Applied patch of type __application/strategic-merge-patch+json__ to target __kubeletconfiguration__" L45: "cluster.go:125: [kubelet-start] Writing kubelet configuration to file __/var/lib/kubelet/config.yaml__" L46: "cluster.go:125: [kubelet-start] Starting the kubelet" L47: "cluster.go:125: [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory __/etc/kubernetes/manifests__" L48: "cluster.go:125: [kubelet-check] Waiting for a healthy kubelet at http://127.0.0.1:10248/healthz. This can take up to 4m0s" L49: "cluster.go:125: [kubelet-check] The kubelet is healthy after 1.27372ms" L50: "cluster.go:125: [control-plane-check] Waiting for healthy control plane components. This can take up to 30m0s" L51: "cluster.go:125: [control-plane-check] Checking kube-apiserver at https://10.0.0.27:6443/livez" L52: "cluster.go:125: [control-plane-check] Checking kube-controller-manager at https://127.0.0.1:10257/healthz" L53: "cluster.go:125: [control-plane-check] Checking kube-scheduler at https://127.0.0.1:10259/livez" L54: "cluster.go:125: [control-plane-check] kube-scheduler is healthy after 12.606401ms" L55: "cluster.go:125: [control-plane-check] kube-controller-manager is healthy after 11.86219ms" L56: "cluster.go:125: [control-plane-check] kube-apiserver is healthy after 1.501145337s" L57: "cluster.go:125: [upload-config] Storing the configuration used in ConfigMap __kubeadm-config__ in the __kube-system__ Namespace" L58: "cluster.go:125: [kubelet] Creating a ConfigMap __kubelet-config__ in namespace kube-system with the configuration for the kubelets in the cluster" L59: "cluster.go:125: [upload-certs] Skipping phase. Please see --upload-certs" L60: "cluster.go:125: [mark-control-plane] Marking the node ci-4081.3.9-n-2df587b7ea as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-lo?ad-balancers]" L61: "cluster.go:125: [mark-control-plane] Marking the node ci-4081.3.9-n-2df587b7ea as control-plane by adding the taints [node-role.kubernetes.io/control-plane:NoSchedule]" L62: "cluster.go:125: [bootstrap-token] Using token: adqa15.avr8537b0bvsxnso" L63: "cluster.go:125: [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles" L64: "cluster.go:125: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes" L65: "cluster.go:125: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials" L66: "cluster.go:125: [bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token" L67: "cluster.go:125: [bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster" L68: "cluster.go:125: [bootstrap-token] Configured RBAC rules to allow the API server kubelet client certificate to access the kubelet API" L69: "cluster.go:125: [bootstrap-token] Creating the __cluster-info__ ConfigMap in the __kube-public__ namespace" L70: "cluster.go:125: [kubelet-finalize] Updating __/etc/kubernetes/kubelet.conf__ to point to a rotatable kubelet client certificate and key" L71: "cluster.go:125: [addons] Applied essential addon: CoreDNS" L72: "cluster.go:125: [addons] Applied essential addon: kube-proxy" L73: "cluster.go:125: " L74: "cluster.go:125: Your Kubernetes control-plane has initialized successfully!" L75: "cluster.go:125: " L76: "cluster.go:125: To start using your cluster, you need to run the following as a regular user:" L77: "cluster.go:125: " L78: "cluster.go:125: mkdir -p $HOME/.kube" L79: "cluster.go:125: sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config" L80: "cluster.go:125: sudo chown $(id -u):$(id -g) $HOME/.kube/config" L81: "cluster.go:125: " L82: "cluster.go:125: Alternatively, if you are the root user, you can run:" L83: "cluster.go:125: " L84: "cluster.go:125: export KUBECONFIG=/etc/kubernetes/admin.conf" L85: "cluster.go:125: " L86: "cluster.go:125: You should now deploy a pod network to the cluster." L87: "cluster.go:125: Run __kubectl apply -f [podnetwork].yaml__ with one of the options listed at:" L88: "cluster.go:125: https://kubernetes.io/docs/concepts/cluster-administration/addons/" L89: "cluster.go:125: " L90: "cluster.go:125: Then you can join any number of worker nodes by running the following on each as root:" L91: "cluster.go:125: " L92: "cluster.go:125: kubeadm join 10.0.0.27:6443 --token adqa15.avr8537b0bvsxnso _" L93: "cluster.go:125: --discovery-token-ca-cert-hash sha256:590c73f5968cec87ba9518e1b8f130891fdbc86f55206db70092e6c36870b0fa " L94: "cluster.go:125: namespace/tigera-operator created" L95: "cluster.go:125: serviceaccount/tigera-operator created" L96: "cluster.go:125: clusterrole.rbac.authorization.k8s.io/tigera-operator-secrets created" L97: "cluster.go:125: clusterrole.rbac.authorization.k8s.io/tigera-operator created" L98: "cluster.go:125: clusterrolebinding.rbac.authorization.k8s.io/tigera-operator created" L99: "cluster.go:125: rolebinding.rbac.authorization.k8s.io/tigera-operator-secrets created" L100: "cluster.go:125: deployment.apps/tigera-operator created" L101: "cluster.go:125: customresourcedefinition.apiextensions.k8s.io/installations.operator.tigera.io condition met" L102: "cluster.go:125: customresourcedefinition.apiextensions.k8s.io/installations.operator.tigera.io condition met" L103: "cluster.go:125: customresourcedefinition.apiextensions.k8s.io/apiservers.operator.tigera.io condition met" L104: "cluster.go:125: customresourcedefinition.apiextensions.k8s.io/apiservers.operator.tigera.io condition met" L105: "cluster.go:125: installation.operator.tigera.io/default created" L106: "cluster.go:125: apiserver.operator.tigera.io/default created" L107: "cluster.go:125: goldmane.operator.tigera.io/default created" L108: "cluster.go:125: whisker.operator.tigera.io/default created" L109: "kubeadm.go:197: unable to setup cluster: unable to create worker node: PUT https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kola-cluster-image-922124aacf/?providers/Microsoft.Compute/virtualMachines/ci-4081.3.9-n-db22003b9d" L110: "--------------------------------------------------------------------------------" L111: "RESPONSE 409: 409 Conflict" L112: "ERROR CODE: OperationNotAllowed" L113: "--------------------------------------------------------------------------------" L114: "{" L115: "__error__: {" L116: "__code__: __OperationNotAllowed__," L117: "__message__: __Operation could not be completed as it results in exceeding approved standardDSv4Family Cores quota. Additional details - Deployment Model: Resource Manager, Location: northeurope, Curr?ent Limit: 65, Current Usage: 64, Additional Required: 2, (Minimum) New Limit Required: 66. Setup Alerts when Quota reaches threshold. Learn more at https://aka.ms/quotamonitoringalerting . Submit a r?equest for Quota increase at https://aka.ms/ProdportalCRP/#blade/Microsoft_Azure_Capacity/UsageAndQuota.ReactView/Parameters/%7B%22subscriptionId%22:%220e46bd28-a80f-4d3a-8200-d9eb8d80cb2e%22,%22comma?nd%22:%22openQuotaApprovalBlade%22,%22quotas%22:[%7B%22location%22:%22northeurope%22,%22providerId%22:%22Microsoft.Compute%22,%22resourceName%22:%22standardDSv4Family%22,%22quotaRequest%22:%7B%22prope?rties%22:%7B%22limit%22:66,%22unit%22:%22Count%22,%22name%22:%7B%22value%22:%22standardDSv4Family%22%7D%7D%7D%7D]%7D by specifying parameters listed in the ???Details??? section for deployment to succ?eed. Please read more about quota limits at https://docs.microsoft.com/en-us/azure/azure-supportability/per-vm-quota-requests__" L118: "}" L119: "}" L120: "--------------------------------------------------------------------------------_" L121: " " ```                 Diagnostic output for azure, run 1 ``` L1: " Error: _cluster.go:125: [config/images] Pulled registry.k8s.io/kube-apiserver:v1.36.3" L2: "cluster.go:125: [config/images] Pulled registry.k8s.io/kube-controller-manager:v1.36.3" L3: "cluster.go:125: [config/images] Pulled registry.k8s.io/kube-scheduler:v1.36.3" L4: "cluster.go:125: [config/images] Pulled registry.k8s.io/kube-proxy:v1.36.3" L5: "cluster.go:125: [config/images] Pulled registry.k8s.io/coredns/coredns:v1.14.2" L6: "cluster.go:125: [config/images] Pulled registry.k8s.io/pause:3.10.2" L7: "cluster.go:125: [config/images] Pulled registry.k8s.io/etcd:3.6.8-0" L8: "cluster.go:125: [init] Using Kubernetes version: v1.36.3" L9: "cluster.go:125: [preflight] Running pre-flight checks" L10: "cluster.go:125: E0724 23:43:00.784041 2729 remote_runtime.go:1177] __RuntimeConfig from runtime service failed__ err=__rpc error: code = Unimplemented desc = unknown method RuntimeConfig for servic?e runtime.v1.RuntimeService__" L11: "cluster.go:125: [WARNING ContainerRuntimeVersion]: You must update your container runtime to a version that supports the CRI method RuntimeConfig. Falling back to using cgroupDriver from kubelet conf?ig will be removed in 1.37. For more information, see https://git.k8s.io/enhancements/keps/sig-node/4033-group-driver-detection-over-cri" L12: "cluster.go:125: [WARNING Hostname]: hostname __ci-4081.3.9-n-996955222c__ could not be reached" L13: "cluster.go:125: [WARNING Hostname]: hostname __ci-4081.3.9-n-996955222c__: lookup ci-4081.3.9-n-996955222c on 168.63.129.16:53: no such host" L14: "cluster.go:125: [WARNING Service-kubelet]: kubelet service is not enabled, please run _systemctl enable kubelet.service_" L15: "cluster.go:125: [preflight] Pulling images required for setting up a Kubernetes cluster" L16: "cluster.go:125: [preflight] This might take a minute or two, depending on the speed of your internet connection" L17: "cluster.go:125: [preflight] You can also perform this action beforehand using _kubeadm config images pull_" L18: "cluster.go:125: W0724 23:43:00.918949 2729 checks.go:907] detected that the sandbox image __registry.k8s.io/pause:3.8__ of the container runtime is inconsistent with that used by kubeadm. It is rec?ommended to use __registry.k8s.io/pause:3.10.2__ as the CRI sandbox image." L19: "cluster.go:125: [certs] Using certificateDir folder __/etc/kubernetes/pki__" L20: "cluster.go:125: [certs] Generating __ca__ certificate and key" L21: "cluster.go:125: [certs] Generating __apiserver__ certificate and key" L22: "cluster.go:125: [certs] apiserver serving cert is signed for DNS names [ci-4081.3.9-n-996955222c kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.?96.0.1 10.0.0.34]" L23: "cluster.go:125: [certs] Generating __apiserver-kubelet-client__ certificate and key" L24: "cluster.go:125: [certs] Generating __front-proxy-ca__ certificate and key" L25: "cluster.go:125: [certs] Generating __front-proxy-client__ certificate and key" L26: "cluster.go:125: [certs] External etcd mode: Skipping etcd/ca certificate authority generation" L27: "cluster.go:125: [certs] External etcd mode: Skipping etcd/server certificate generation" L28: "cluster.go:125: [certs] External etcd mode: Skipping etcd/peer certificate generation" L29: "cluster.go:125: [certs] External etcd mode: Skipping etcd/healthcheck-client certificate generation" L30: "cluster.go:125: [certs] External etcd mode: Skipping apiserver-etcd-client certificate generation" L31: "cluster.go:125: [certs] Generating __sa__ key and public key" L32: "cluster.go:125: [kubeconfig] Using kubeconfig folder __/etc/kubernetes__" L33: "cluster.go:125: [kubeconfig] Writing __admin.conf__ kubeconfig file" L34: "cluster.go:125: [kubeconfig] Writing __super-admin.conf__ kubeconfig file" L35: "cluster.go:125: [kubeconfig] Writing __kubelet.conf__ kubeconfig file" L36: "cluster.go:125: [kubeconfig] Writing __controller-manager.conf__ kubeconfig file" L37: "cluster.go:125: [kubeconfig] Writing __scheduler.conf__ kubeconfig file" L38: "cluster.go:125: [control-plane] Using manifest folder __/etc/kubernetes/manifests__" L39: "cluster.go:125: [control-plane] Creating static Pod manifest for __kube-apiserver__" L40: "cluster.go:125: [control-plane] Creating static Pod manifest for __kube-controller-manager__" L41: "cluster.go:125: [control-plane] Creating static Pod manifest for __kube-scheduler__" L42: "cluster.go:125: [kubelet-start] Writing kubelet environment file with flags to file __/var/lib/kubelet/kubeadm-flags.env__" L43: "cluster.go:125: [kubelet-start] Writing kubelet configuration to file __/var/lib/kubelet/instance-config.yaml__" L44: "cluster.go:125: [patches] Applied patch of type __application/strategic-merge-patch+json__ to target __kubeletconfiguration__" L45: "cluster.go:125: [kubelet-start] Writing kubelet configuration to file __/var/lib/kubelet/config.yaml__" L46: "cluster.go:125: [kubelet-start] Starting the kubelet" L47: "cluster.go:125: [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory __/etc/kubernetes/manifests__" L48: "cluster.go:125: [kubelet-check] Waiting for a healthy kubelet at http://127.0.0.1:10248/healthz. This can take up to 4m0s" L49: "cluster.go:125: [kubelet-check] The kubelet is healthy after 511.707??s" L50: "cluster.go:125: [control-plane-check] Waiting for healthy control plane components. This can take up to 30m0s" L51: "cluster.go:125: [control-plane-check] Checking kube-apiserver at https://10.0.0.34:6443/livez" L52: "cluster.go:125: [control-plane-check] Checking kube-controller-manager at https://127.0.0.1:10257/healthz" L53: "cluster.go:125: [control-plane-check] Checking kube-scheduler at https://127.0.0.1:10259/livez" L54: "cluster.go:125: [control-plane-check] kube-controller-manager is healthy after 6.195287ms" L55: "cluster.go:125: [control-plane-check] kube-scheduler is healthy after 6.955298ms" L56: "cluster.go:125: [control-plane-check] kube-apiserver is healthy after 2.002087485s" L57: "cluster.go:125: [upload-config] Storing the configuration used in ConfigMap __kubeadm-config__ in the __kube-system__ Namespace" L58: "cluster.go:125: [kubelet] Creating a ConfigMap __kubelet-config__ in namespace kube-system with the configuration for the kubelets in the cluster" L59: "cluster.go:125: [upload-certs] Skipping phase. Please see --upload-certs" L60: "cluster.go:125: [mark-control-plane] Marking the node ci-4081.3.9-n-996955222c as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-lo?ad-balancers]" L61: "cluster.go:125: [mark-control-plane] Marking the node ci-4081.3.9-n-996955222c as control-plane by adding the taints [node-role.kubernetes.io/control-plane:NoSchedule]" L62: "cluster.go:125: [bootstrap-token] Using token: m0d4fd.wwky0ftj4z14l9l5" L63: "cluster.go:125: [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles" L64: "cluster.go:125: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes" L65: "cluster.go:125: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials" L66: "cluster.go:125: [bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token" L67: "cluster.go:125: [bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster" L68: "cluster.go:125: [bootstrap-token] Configured RBAC rules to allow the API server kubelet client certificate to access the kubelet API" L69: "cluster.go:125: [bootstrap-token] Creating the __cluster-info__ ConfigMap in the __kube-public__ namespace" L70: "cluster.go:125: [kubelet-finalize] Updating __/etc/kubernetes/kubelet.conf__ to point to a rotatable kubelet client certificate and key" L71: "cluster.go:125: [addons] Applied essential addon: CoreDNS" L72: "cluster.go:125: [addons] Applied essential addon: kube-proxy" L73: "cluster.go:125: " L74: "cluster.go:125: Your Kubernetes control-plane has initialized successfully!" L75: "cluster.go:125: " L76: "cluster.go:125: To start using your cluster, you need to run the following as a regular user:" L77: "cluster.go:125: " L78: "cluster.go:125: mkdir -p $HOME/.kube" L79: "cluster.go:125: sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config" L80: "cluster.go:125: sudo chown $(id -u):$(id -g) $HOME/.kube/config" L81: "cluster.go:125: " L82: "cluster.go:125: Alternatively, if you are the root user, you can run:" L83: "cluster.go:125: " L84: "cluster.go:125: export KUBECONFIG=/etc/kubernetes/admin.conf" L85: "cluster.go:125: " L86: "cluster.go:125: You should now deploy a pod network to the cluster." L87: "cluster.go:125: Run __kubectl apply -f [podnetwork].yaml__ with one of the options listed at:" L88: "cluster.go:125: https://kubernetes.io/docs/concepts/cluster-administration/addons/" L89: "cluster.go:125: " L90: "cluster.go:125: Then you can join any number of worker nodes by running the following on each as root:" L91: "cluster.go:125: " L92: "cluster.go:125: kubeadm join 10.0.0.34:6443 --token m0d4fd.wwky0ftj4z14l9l5 _" L93: "cluster.go:125: --discovery-token-ca-cert-hash sha256:497e9fc01dc9e674103c3b7a1132b25e3903d3464661b9da72d195aab4a733c7 " L94: "cluster.go:125: namespace/tigera-operator created" L95: "cluster.go:125: serviceaccount/tigera-operator created" L96: "cluster.go:125: clusterrole.rbac.authorization.k8s.io/tigera-operator-secrets created" L97: "cluster.go:125: clusterrole.rbac.authorization.k8s.io/tigera-operator created" L98: "cluster.go:125: clusterrolebinding.rbac.authorization.k8s.io/tigera-operator created" L99: "cluster.go:125: rolebinding.rbac.authorization.k8s.io/tigera-operator-secrets created" L100: "cluster.go:125: deployment.apps/tigera-operator created" L101: "cluster.go:125: customresourcedefinition.apiextensions.k8s.io/installations.operator.tigera.io condition met" L102: "cluster.go:125: customresourcedefinition.apiextensions.k8s.io/installations.operator.tigera.io condition met" L103: "cluster.go:125: customresourcedefinition.apiextensions.k8s.io/apiservers.operator.tigera.io condition met" L104: "cluster.go:125: customresourcedefinition.apiextensions.k8s.io/apiservers.operator.tigera.io condition met" L105: "cluster.go:125: installation.operator.tigera.io/default created" L106: "cluster.go:125: apiserver.operator.tigera.io/default created" L107: "cluster.go:125: goldmane.operator.tigera.io/default created" L108: "cluster.go:125: whisker.operator.tigera.io/default created" L109: "kubeadm.go:197: unable to setup cluster: unable to create worker node: PollUntilDone(ci-4081.3.9-n-322cffbb6c): GET https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/prov?iders/Microsoft.Compute/locations/northeurope/operations/931e47fd-767e-4a13-88a4-89c875bd856a" L110: "--------------------------------------------------------------------------------" L111: "RESPONSE 200: 200 OK" L112: "ERROR CODE: OperationPreempted" L113: "--------------------------------------------------------------------------------" L114: "{" L115: "__startTime__: __2026-07-24T23:43:28.0605223+00:00__," L116: "__endTime__: __2026-07-24T23:44:56.2596252+00:00__," L117: "__status__: __Canceled__," L118: "__error__: {" L119: "__code__: __OperationPreempted__," L120: "__message__: __Operation execution has been preempted by a more recent operation.__" L121: "}," L122: "__name__: __931e47fd-767e-4a13-88a4-89c875bd856a__" L123: "}" L124: "--------------------------------------------------------------------------------_" L125: " " ```
🟢 ok **kubeadm.v1.36.1.cilium.base**; Succeeded: azure (3); Failed: azure (1, 2)
                Diagnostic output for azure, run 2 ``` L1: " Error: _kubeadm.go:197: unable to setup cluster: unable to create master node: PUT https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kola-cluster-image-922?124aacf/providers/Microsoft.Compute/virtualMachines/ci-4081.3.9-n-406a677240" L2: "--------------------------------------------------------------------------------" L3: "RESPONSE 409: 409 Conflict" L4: "ERROR CODE: OperationNotAllowed" L5: "--------------------------------------------------------------------------------" L6: "{" L7: "__error__: {" L8: "__code__: __OperationNotAllowed__," L9: "__message__: __Operation could not be completed as it results in exceeding approved standardDSv4Family Cores quota. Additional details - Deployment Model: Resource Manager, Location: northeurope, Curr?ent Limit: 65, Current Usage: 64, Additional Required: 2, (Minimum) New Limit Required: 66. Setup Alerts when Quota reaches threshold. Learn more at https://aka.ms/quotamonitoringalerting . Submit a r?equest for Quota increase at https://aka.ms/ProdportalCRP/#blade/Microsoft_Azure_Capacity/UsageAndQuota.ReactView/Parameters/%7B%22subscriptionId%22:%220e46bd28-a80f-4d3a-8200-d9eb8d80cb2e%22,%22comma?nd%22:%22openQuotaApprovalBlade%22,%22quotas%22:[%7B%22location%22:%22northeurope%22,%22providerId%22:%22Microsoft.Compute%22,%22resourceName%22:%22standardDSv4Family%22,%22quotaRequest%22:%7B%22prope?rties%22:%7B%22limit%22:66,%22unit%22:%22Count%22,%22name%22:%7B%22value%22:%22standardDSv4Family%22%7D%7D%7D%7D]%7D by specifying parameters listed in the ???Details??? section for deployment to succ?eed. Please read more about quota limits at https://docs.microsoft.com/en-us/azure/azure-supportability/per-vm-quota-requests__" L10: "}" L11: "}" L12: "--------------------------------------------------------------------------------_" L13: " " ```                 Diagnostic output for azure, run 1 ``` L1: " Error: _cluster.go:125: [config/images] Pulled registry.k8s.io/kube-apiserver:v1.36.3" L2: "cluster.go:125: [config/images] Pulled registry.k8s.io/kube-controller-manager:v1.36.3" L3: "cluster.go:125: [config/images] Pulled registry.k8s.io/kube-scheduler:v1.36.3" L4: "cluster.go:125: [config/images] Pulled registry.k8s.io/kube-proxy:v1.36.3" L5: "cluster.go:125: [config/images] Pulled registry.k8s.io/coredns/coredns:v1.14.2" L6: "cluster.go:125: [config/images] Pulled registry.k8s.io/pause:3.10.2" L7: "cluster.go:125: [config/images] Pulled registry.k8s.io/etcd:3.6.8-0" L8: "cluster.go:125: [init] Using Kubernetes version: v1.36.3" L9: "cluster.go:125: [preflight] Running pre-flight checks" L10: "cluster.go:125: E0724 23:42:01.204592 2577 remote_runtime.go:1177] __RuntimeConfig from runtime service failed__ err=__rpc error: code = Unimplemented desc = unknown method RuntimeConfig for servic?e runtime.v1.RuntimeService__" L11: "cluster.go:125: [WARNING ContainerRuntimeVersion]: You must update your container runtime to a version that supports the CRI method RuntimeConfig. Falling back to using cgroupDriver from kubelet conf?ig will be removed in 1.37. For more information, see https://git.k8s.io/enhancements/keps/sig-node/4033-group-driver-detection-over-cri" L12: "cluster.go:125: [WARNING Hostname]: hostname __ci-4081.3.9-n-cb5aabc4f5__ could not be reached" L13: "cluster.go:125: [WARNING Hostname]: hostname __ci-4081.3.9-n-cb5aabc4f5__: lookup ci-4081.3.9-n-cb5aabc4f5 on 168.63.129.16:53: no such host" L14: "cluster.go:125: [WARNING Service-kubelet]: kubelet service is not enabled, please run _systemctl enable kubelet.service_" L15: "cluster.go:125: [preflight] Pulling images required for setting up a Kubernetes cluster" L16: "cluster.go:125: [preflight] This might take a minute or two, depending on the speed of your internet connection" L17: "cluster.go:125: [preflight] You can also perform this action beforehand using _kubeadm config images pull_" L18: "cluster.go:125: W0724 23:42:01.364597 2577 checks.go:907] detected that the sandbox image __registry.k8s.io/pause:3.8__ of the container runtime is inconsistent with that used by kubeadm. It is rec?ommended to use __registry.k8s.io/pause:3.10.2__ as the CRI sandbox image." L19: "cluster.go:125: [certs] Using certificateDir folder __/etc/kubernetes/pki__" L20: "cluster.go:125: [certs] Generating __ca__ certificate and key" L21: "cluster.go:125: [certs] Generating __apiserver__ certificate and key" L22: "cluster.go:125: [certs] apiserver serving cert is signed for DNS names [ci-4081.3.9-n-cb5aabc4f5 kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.?96.0.1 10.0.0.33]" L23: "cluster.go:125: [certs] Generating __apiserver-kubelet-client__ certificate and key" L24: "cluster.go:125: [certs] Generating __front-proxy-ca__ certificate and key" L25: "cluster.go:125: [certs] Generating __front-proxy-client__ certificate and key" L26: "cluster.go:125: [certs] External etcd mode: Skipping etcd/ca certificate authority generation" L27: "cluster.go:125: [certs] External etcd mode: Skipping etcd/server certificate generation" L28: "cluster.go:125: [certs] External etcd mode: Skipping etcd/peer certificate generation" L29: "cluster.go:125: [certs] External etcd mode: Skipping etcd/healthcheck-client certificate generation" L30: "cluster.go:125: [certs] External etcd mode: Skipping apiserver-etcd-client certificate generation" L31: "cluster.go:125: [certs] Generating __sa__ key and public key" L32: "cluster.go:125: [kubeconfig] Using kubeconfig folder __/etc/kubernetes__" L33: "cluster.go:125: [kubeconfig] Writing __admin.conf__ kubeconfig file" L34: "cluster.go:125: [kubeconfig] Writing __super-admin.conf__ kubeconfig file" L35: "cluster.go:125: [kubeconfig] Writing __kubelet.conf__ kubeconfig file" L36: "cluster.go:125: [kubeconfig] Writing __controller-manager.conf__ kubeconfig file" L37: "cluster.go:125: [kubeconfig] Writing __scheduler.conf__ kubeconfig file" L38: "cluster.go:125: [control-plane] Using manifest folder __/etc/kubernetes/manifests__" L39: "cluster.go:125: [control-plane] Creating static Pod manifest for __kube-apiserver__" L40: "cluster.go:125: [control-plane] Creating static Pod manifest for __kube-controller-manager__" L41: "cluster.go:125: [control-plane] Creating static Pod manifest for __kube-scheduler__" L42: "cluster.go:125: [kubelet-start] Writing kubelet environment file with flags to file __/var/lib/kubelet/kubeadm-flags.env__" L43: "cluster.go:125: [kubelet-start] Writing kubelet configuration to file __/var/lib/kubelet/instance-config.yaml__" L44: "cluster.go:125: [patches] Applied patch of type __application/strategic-merge-patch+json__ to target __kubeletconfiguration__" L45: "cluster.go:125: [kubelet-start] Writing kubelet configuration to file __/var/lib/kubelet/config.yaml__" L46: "cluster.go:125: [kubelet-start] Starting the kubelet" L47: "cluster.go:125: [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory __/etc/kubernetes/manifests__" L48: "cluster.go:125: [kubelet-check] Waiting for a healthy kubelet at http://127.0.0.1:10248/healthz. This can take up to 4m0s" L49: "cluster.go:125: [kubelet-check] The kubelet is healthy after 2.057914ms" L50: "cluster.go:125: [control-plane-check] Waiting for healthy control plane components. This can take up to 30m0s" L51: "cluster.go:125: [control-plane-check] Checking kube-apiserver at https://10.0.0.33:6443/livez" L52: "cluster.go:125: [control-plane-check] Checking kube-controller-manager at https://127.0.0.1:10257/healthz" L53: "cluster.go:125: [control-plane-check] Checking kube-scheduler at https://127.0.0.1:10259/livez" L54: "cluster.go:125: [control-plane-check] kube-scheduler is healthy after 7.743953ms" L55: "cluster.go:125: [control-plane-check] kube-controller-manager is healthy after 10.722372ms" L56: "cluster.go:125: [control-plane-check] kube-apiserver is healthy after 1.50232965s" L57: "cluster.go:125: [upload-config] Storing the configuration used in ConfigMap __kubeadm-config__ in the __kube-system__ Namespace" L58: "cluster.go:125: [kubelet] Creating a ConfigMap __kubelet-config__ in namespace kube-system with the configuration for the kubelets in the cluster" L59: "cluster.go:125: [upload-certs] Skipping phase. Please see --upload-certs" L60: "cluster.go:125: [mark-control-plane] Marking the node ci-4081.3.9-n-cb5aabc4f5 as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-lo?ad-balancers]" L61: "cluster.go:125: [mark-control-plane] Marking the node ci-4081.3.9-n-cb5aabc4f5 as control-plane by adding the taints [node-role.kubernetes.io/control-plane:NoSchedule]" L62: "cluster.go:125: [bootstrap-token] Using token: afn4y7.vywqsvkfh70tdsor" L63: "cluster.go:125: [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles" L64: "cluster.go:125: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes" L65: "cluster.go:125: [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials" L66: "cluster.go:125: [bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token" L67: "cluster.go:125: [bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster" L68: "cluster.go:125: [bootstrap-token] Configured RBAC rules to allow the API server kubelet client certificate to access the kubelet API" L69: "cluster.go:125: [bootstrap-token] Creating the __cluster-info__ ConfigMap in the __kube-public__ namespace" L70: "cluster.go:125: [kubelet-finalize] Updating __/etc/kubernetes/kubelet.conf__ to point to a rotatable kubelet client certificate and key" L71: "cluster.go:125: [addons] Applied essential addon: CoreDNS" L72: "cluster.go:125: [addons] Applied essential addon: kube-proxy" L73: "cluster.go:125: " L74: "cluster.go:125: Your Kubernetes control-plane has initialized successfully!" L75: "cluster.go:125: " L76: "cluster.go:125: To start using your cluster, you need to run the following as a regular user:" L77: "cluster.go:125: " L78: "cluster.go:125: mkdir -p $HOME/.kube" L79: "cluster.go:125: sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config" L80: "cluster.go:125: sudo chown $(id -u):$(id -g) $HOME/.kube/config" L81: "cluster.go:125: " L82: "cluster.go:125: Alternatively, if you are the root user, you can run:" L83: "cluster.go:125: " L84: "cluster.go:125: export KUBECONFIG=/etc/kubernetes/admin.conf" L85: "cluster.go:125: " L86: "cluster.go:125: You should now deploy a pod network to the cluster." L87: "cluster.go:125: Run __kubectl apply -f [podnetwork].yaml__ with one of the options listed at:" L88: "cluster.go:125: https://kubernetes.io/docs/concepts/cluster-administration/addons/" L89: "cluster.go:125: " L90: "cluster.go:125: Then you can join any number of worker nodes by running the following on each as root:" L91: "cluster.go:125: " L92: "cluster.go:125: kubeadm join 10.0.0.33:6443 --token afn4y7.vywqsvkfh70tdsor _" L93: "cluster.go:125: --discovery-token-ca-cert-hash sha256:e024fd27ebc2fdc8b97a05e6cd875aca37b910711ccf07f6d204faa5319ff27a " L94: "cluster.go:125: i Using Cilium version 1.12.5" L95: "cluster.go:125: ? Auto-detected cluster name: kubernetes" L96: "cluster.go:125: ? Auto-detected datapath mode: tunnel" L97: "cluster.go:125: ? Auto-detected kube-proxy has been installed" L98: "cluster.go:125: i helm template --namespace kube-system cilium cilium/cilium --version 1.12.5 --set cluster.id=0,cluster.name=kubernetes,encryption.nodeEncryption=false,extraConfig.cluster-pool-ipv4-?cidr=192.168.0.0/17,extraConfig.enable-endpoint-routes=true,kubeProxyReplacement=disabled,operator.replicas=1,serviceAccounts.cilium.name=cilium,serviceAccounts.operator.name=cilium-operator,tunnel=vx?lan" L99: "cluster.go:125: i Storing helm values file in kube-system/cilium-cli-helm-values Secret" L100: "cluster.go:125: ? Created CA in secret cilium-ca" L101: "cluster.go:125: ? Generating certificates for Hubble..." L102: "cluster.go:125: ? Creating Service accounts..." L103: "cluster.go:125: ? Creating Cluster roles..." L104: "cluster.go:125: ? Creating ConfigMap for Cilium version 1.12.5..." L105: "cluster.go:125: i Manual overwrite in ConfigMap: enable-endpoint-routes=true" L106: "cluster.go:125: i Manual overwrite in ConfigMap: cluster-pool-ipv4-cidr=192.168.0.0/17" L107: "cluster.go:125: ? Creating Agent DaemonSet..." L108: "cluster.go:125: level=warning msg=__spec.template.metadata.annotations[container.apparmor.security.beta.kubernetes.io/mount-cgroup]: deprecated since v1.30; use the ___appArmorProfile___ field instead?__ subsys=klog" L109: "cluster.go:125: level=warning msg=__spec.template.metadata.annotations[container.apparmor.security.beta.kubernetes.io/apply-sysctl-overwrites]: deprecated since v1.30; use the ___appArmorProfile___ fi?eld instead__ subsys=klog" L110: "cluster.go:125: level=warning msg=__spec.template.metadata.annotations[container.apparmor.security.beta.kubernetes.io/clean-cilium-state]: deprecated since v1.30; use the ___appArmorProfile___ field i?nstead__ subsys=klog" L111: "cluster.go:125: level=warning msg=__spec.template.metadata.annotations[container.apparmor.security.beta.kubernetes.io/cilium-agent]: deprecated since v1.30; use the ___appArmorProfile___ field instead?__ subsys=klog" L112: "cluster.go:125: ? Creating Operator Deployment..." L113: "cluster.go:125: ? Waiting for Cilium to be installed and ready..." L114: "cluster.go:125: ? Cilium was successfully installed! Run _cilium status_ to view installation health" L115: "cluster.go:125: ?[33m /??_" L116: "cluster.go:125: ?[36m /???[33m___/?[32m??_?[0m Cilium: ?[32mOK?[0m" L117: "cluster.go:125: ?[36m ___?[31m/??_?[32m__/?[0m Operator: ?[32mOK?[0m" L118: "cluster.go:125: ?[32m /???[31m___/?[35m??_?[0m Hubble: ?[36mdisabled?[0m" L119: "cluster.go:125: ?[32m ___?[34m/??_?[35m__/?[0m ClusterMesh: ?[36mdisabled?[0m" L120: "cluster.go:125: ?[34m ___/" L121: "cluster.go:125: ?[0m" L122: "cluster.go:125: Deployment cilium-operator " L123: "cluster.go:125: DaemonSet cilium " L124: "cluster.go:125: Containers: cilium " L125: "cluster.go:125: cilium-operator " L126: "cluster.go:125: Cluster Pods: 0/0 managed by Cilium" L127: "kubeadm.go:197: unable to setup cluster: unable to create worker node: PUT https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kola-cluster-image-b4ba8baeca/?providers/Microsoft.Compute/virtualMachines/ci-4081.3.9-n-0e0ea4854e" L128: "--------------------------------------------------------------------------------" L129: "RESPONSE 409: 409 Conflict" L130: "ERROR CODE: OperationNotAllowed" L131: "--------------------------------------------------------------------------------" L132: "{" L133: "__error__: {" L134: "__code__: __OperationNotAllowed__," L135: "__message__: __Operation could not be completed as it results in exceeding approved standardDSv4Family Cores quota. Additional details - Deployment Model: Resource Manager, Location: northeurope, Curr?ent Limit: 65, Current Usage: 64, Additional Required: 2, (Minimum) New Limit Required: 66. Setup Alerts when Quota reaches threshold. Learn more at https://aka.ms/quotamonitoringalerting . Submit a r?equest for Quota increase at https://aka.ms/ProdportalCRP/#blade/Microsoft_Azure_Capacity/UsageAndQuota.ReactView/Parameters/%7B%22subscriptionId%22:%220e46bd28-a80f-4d3a-8200-d9eb8d80cb2e%22,%22comma?nd%22:%22openQuotaApprovalBlade%22,%22quotas%22:[%7B%22location%22:%22northeurope%22,%22providerId%22:%22Microsoft.Compute%22,%22resourceName%22:%22standardDSv4Family%22,%22quotaRequest%22:%7B%22prope?rties%22:%7B%22limit%22:66,%22unit%22:%22Count%22,%22name%22:%7B%22value%22:%22standardDSv4Family%22%7D%7D%7D%7D]%7D by specifying parameters listed in the ???Details??? section for deployment to succ?eed. Please read more about quota limits at https://docs.microsoft.com/en-us/azure/azure-supportability/per-vm-quota-requests__" L136: "}" L137: "}" L138: "--------------------------------------------------------------------------------_" L139: " " ```
🟢 ok **kubeadm.v1.36.1.flannel.base**; Succeeded: azure (2); Failed: azure (1)
                Diagnostic output for azure, run 1 ``` L1: " Error: _kubeadm.go:197: unable to setup cluster: unable to create etcd node: PollUntilDone(ci-4081.3.9-n-66bc99e503): GET https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2?e/providers/Microsoft.Compute/locations/northeurope/operations/3f2c3f67-dc9f-4fb0-83f6-6106e504853e" L2: "--------------------------------------------------------------------------------" L3: "RESPONSE 200: 200 OK" L4: "ERROR CODE: OperationPreempted" L5: "--------------------------------------------------------------------------------" L6: "{" L7: "__startTime__: __2026-07-24T23:44:04.0112833+00:00__," L8: "__endTime__: __2026-07-24T23:45:24.5273099+00:00__," L9: "__status__: __Canceled__," L10: "__error__: {" L11: "__code__: __OperationPreempted__," L12: "__message__: __Operation execution has been preempted by a more recent operation.__" L13: "}," L14: "__name__: __3f2c3f67-dc9f-4fb0-83f6-6106e504853e__" L15: "}" L16: "--------------------------------------------------------------------------------_" L17: " " ```
🟢 ok **sysext.custom-docker.sysext**; Succeeded: azure (2); Failed: azure (1)
                Diagnostic output for azure, run 1 ``` L1: " Error: _harness.go:646: Cluster failed starting machines: waiting for machine to become active: GET https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kola-?cluster-image-b4ba8baeca/providers/Microsoft.Compute/virtualMachines/ci-4081.3.9-n-076fde0784" L2: "--------------------------------------------------------------------------------" L3: "RESPONSE 404: 404 Not Found" L4: "ERROR CODE: NotFound" L5: "--------------------------------------------------------------------------------" L6: "{" L7: "__error__: {" L8: "__code__: __NotFound__," L9: "__message__: __The entity was not found in this Azure location.__" L10: "}" L11: "}" L12: "--------------------------------------------------------------------------------_" L13: " " ```
🟢 ok **sysext.disable-containerd**; Succeeded: azure (1) 🟢 ok **sysext.disable-docker**; Succeeded: azure (2); Failed: azure (1)
                Diagnostic output for azure, run 1 ``` L1: " Error: _harness.go:646: Cluster failed starting machines: creating public ip: PUT https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kola-cluster-image-b4ba?8baeca/providers/Microsoft.Network/publicIPAddresses/ip-f884a4128a" L2: "--------------------------------------------------------------------------------" L3: "RESPONSE 409: 409 Conflict" L4: "ERROR CODE: ResourceGroupBeingDeleted" L5: "--------------------------------------------------------------------------------" L6: "{" L7: "__error__: {" L8: "__code__: __ResourceGroupBeingDeleted__," L9: "__message__: __The resource group _kola-cluster-image-b4ba8baeca_ is in deprovisioning state and cannot perform this operation.__" L10: "}" L11: "}" L12: "--------------------------------------------------------------------------------_" L13: " " ```
🟢 ok **sysext.simple**; Succeeded: azure (2); Failed: azure (1)
                Diagnostic output for azure, run 1 ``` L1: " Error: _harness.go:646: Cluster failed starting machines: PUT https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kola-cluster-image-b4ba8baeca/providers/Mic?rosoft.Compute/virtualMachines/ci-4081.3.9-n-3f3407d9c6" L2: "--------------------------------------------------------------------------------" L3: "RESPONSE 409: 409 Conflict" L4: "ERROR CODE: OperationNotAllowed" L5: "--------------------------------------------------------------------------------" L6: "{" L7: "__error__: {" L8: "__code__: __OperationNotAllowed__," L9: "__message__: __Operation could not be completed as it results in exceeding approved standardDSv4Family Cores quota. Additional details - Deployment Model: Resource Manager, Location: northeurope, Curr?ent Limit: 65, Current Usage: 64, Additional Required: 2, (Minimum) New Limit Required: 66. Setup Alerts when Quota reaches threshold. Learn more at https://aka.ms/quotamonitoringalerting . Submit a r?equest for Quota increase at https://aka.ms/ProdportalCRP/#blade/Microsoft_Azure_Capacity/UsageAndQuota.ReactView/Parameters/%7B%22subscriptionId%22:%220e46bd28-a80f-4d3a-8200-d9eb8d80cb2e%22,%22comma?nd%22:%22openQuotaApprovalBlade%22,%22quotas%22:[%7B%22location%22:%22northeurope%22,%22providerId%22:%22Microsoft.Compute%22,%22resourceName%22:%22standardDSv4Family%22,%22quotaRequest%22:%7B%22prope?rties%22:%7B%22limit%22:66,%22unit%22:%22Count%22,%22name%22:%7B%22value%22:%22standardDSv4Family%22%7D%7D%7D%7D]%7D by specifying parameters listed in the ???Details??? section for deployment to succ?eed. Please read more about quota limits at https://docs.microsoft.com/en-us/azure/azure-supportability/per-vm-quota-requests__" L10: "}" L11: "}" L12: "--------------------------------------------------------------------------------_" L13: " " ```
🟢 ok **systemd.journal.user**; Succeeded: azure (1) 🟢 ok **systemd.sysusers.gshadow**; Succeeded: azure (2); Failed: azure (1)
                Diagnostic output for azure, run 1 ``` L1: " Error: _harness.go:646: Cluster failed starting machines: PUT https://management.azure.com/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kola-cluster-image-b4ba8baeca/providers/Mic?rosoft.Compute/virtualMachines/ci-4081.3.9-n-b68eab633b" L2: "--------------------------------------------------------------------------------" L3: "RESPONSE 409: 409 Conflict" L4: "ERROR CODE: OperationNotAllowed" L5: "--------------------------------------------------------------------------------" L6: "{" L7: "__error__: {" L8: "__code__: __OperationNotAllowed__," L9: "__message__: __Operation could not be completed as it results in exceeding approved standardDSv4Family Cores quota. Additional details - Deployment Model: Resource Manager, Location: northeurope, Curr?ent Limit: 65, Current Usage: 64, Additional Required: 2, (Minimum) New Limit Required: 66. Setup Alerts when Quota reaches threshold. Learn more at https://aka.ms/quotamonitoringalerting . Submit a r?equest for Quota increase at https://aka.ms/ProdportalCRP/#blade/Microsoft_Azure_Capacity/UsageAndQuota.ReactView/Parameters/%7B%22subscriptionId%22:%220e46bd28-a80f-4d3a-8200-d9eb8d80cb2e%22,%22comma?nd%22:%22openQuotaApprovalBlade%22,%22quotas%22:[%7B%22location%22:%22northeurope%22,%22providerId%22:%22Microsoft.Compute%22,%22resourceName%22:%22standardDSv4Family%22,%22quotaRequest%22:%7B%22prope?rties%22:%7B%22limit%22:66,%22unit%22:%22Count%22,%22name%22:%7B%22value%22:%22standardDSv4Family%22%7D%7D%7D%7D]%7D by specifying parameters listed in the ???Details??? section for deployment to succ?eed. Please read more about quota limits at https://docs.microsoft.com/en-us/azure/azure-supportability/per-vm-quota-requests__" L10: "}" L11: "}" L12: "--------------------------------------------------------------------------------_" L13: " " ```